real-time display on screen with RAW10 sensor at IMX8MP platform

[fengyus]

i have a project with a OV2740 at IMX8M PLUS platform. OV2740's output format is RAW10. actually i have successfully finished the driver development. i can use v4l2 commands to capture images well. but i can't put the sensor's video stream on screen. i want to display the sensor video stream on screen in real time. how can i do that?

[malik_cisse · Accepted solution]

Hi @fengyus 
Your issue is that you have connected your sensor only to ISI which only outputs RAW10(BGGR).
RAW10(BGGR) cannot be displayed unless you process it via ISP first. This is because this is Raw bayer format. 
You need to also activate ISP in device tree and setup the ISP pipeline as described in iMX8MP_CAMERA_DISPLAY_GUIDE.pdf
ISP will convert RAW10(BGGR) to YUYV (which is YUV 4:2:2). This is input to all pipelines described in i.MX8GStreamerUserGuide.pdf (sent previously)
After that you should see two devices in /dev:
/dev/video0 and /dev/video1

[fengyus]

yes it is. thanks a lot.

[Alejandro_Salas]

Hello @fengyus 

You can refer to GStreamer User guide (see attached) in chapter 6.

There is explained how to stream a video from a camera device.

Best regards,

Salas.

[fengyus]

Hi Alejandro_Salas：

when i run "gst-device-monitor-1.0", i get my device as below:

Device found:

name  : mxc-isi-cap

class : Video/Source

caps  : video/x-raw, format=YUY2, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=YUY2, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=NV12, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=NV12, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=NV12, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=NV12, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=BGRA, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=BGRA, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=BGRx, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=BGRx, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=BGR, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=BGR, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=RGB, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=RGB, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

        video/x-raw, format=RGB16, width=1920, height=1080, framerate=60/1

        video/x-raw(format:Interlaced), format=RGB16, width=1920, height=1080, framerate=60/1, interlace-mode=alternate

properties:

udev-probed = true

device.bus_path = platform-32c00000.bus:camera

sysfs.path = /sys/devices/platform/soc@0/32c00000.bus/32c00000.bus:camera/video4linux/video4

device.subsystem = video4linux

device.product.name = mxc-isi-cap

device.capabilities = :capture:

device.api = v4l2

device.path = /dev/video4

v4l2.device.driver = mxc-isi-cap

v4l2.device.card = mxc-isi-cap

v4l2.device.bus_info = platform:32e02000.isi:cap_devic

v4l2.device.version = 393508 (0x00060124)

v4l2.device.capabilities = 2216693760 (0x84201000)

v4l2.device.device_caps = 69210112 (0x04201000)

gst-launch-1.0 v4l2src device=/dev/video4 ! ...

as you can see, gstreamer think the format of my sensor is YUY2、NV12、BGRA、BGRX、BGR or RGB16. but in fact, my format should be RAW10(BGGR). in v4l2 commands, it should be BG10. how can i make gstreamer find my right format?