iMX8: Waylandsink doesn't support UYVY?

[kaartic_sn]

Hi,

I'm using the sumo distribution for iMX8 on a iMX8M Quad EVK. That is the BSP based on kernel version 4.14.78. I tried to stream data from a camera sensor (connected to MIPI) that provides output in UYVY format. I tried to use the following Gstreamer pipeline to stream to the display:

 gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! waylandsink -v
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../../../../git/libs/gst/base/gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.035448601
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I know that the sensor is streaming as I see successful streaming via v4l2-ctl. Further, the following Gstreamer pipeline correctly shows the frame rate for us:

gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! fpsdisplaysink video-sink=fakesink sync=false text-overlay=false -v

So, there's no issue with the sensor. I also checked the output of gst-inspect-1.0 waylandsink and ensured that waylandsink does support the UYVY format.

It's not clear why we get the error noted above. Any help would be very useful!

[joanxie]

waylandsink should support UYVY formats, I tested on my board with gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! waylandsink -v, the result is ok when I connect ov5640, but when I disconnect this camera, I got the same error with you, so pls check if you connect your camera to the cable tight enough, then you can try t use gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! kmssink -v, how about this command? is it ok?

[kaartic_sn]

Hi joanxie‌,

Thank you for the response!

Joan Xie wrote:

waylandsink should support UYVY formats, I tested on my board with gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! waylandsink -v, the result is ok when I connect ov5640, but when I disconnect this camera, I got the same error with you,

I'm also able to stream using waylandsink when I connect the OV5640. I only face the issue when I connected another sensor. Further, from my analysis of the OV5640 driver (ov5640_mipi_v2.c), the output format of OV5640 is YUYV while the output format of the sensor I'm trying to stream is UYVY. I'm not sure why YUYV streams successfully with waylandsink but UYVY does not?

so pls check if you connect your camera to the cable tight enough, then you can try t use gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! kmssink -v, how about this command? is it ok?

I've verified that the camera is connected tight enough. Also, I'm able to stream using kmssink if I kill the wayland GUI i.e., the pipeline you mentioned works. Streaming using v4l2-ctl  works too.

Also, I am able to stream using waylandsink itself if I use videoconvert in between the v4l2src and waylandsink but the frame rates are bad:

gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! videoconvert ! waylandsink -v

Please help us identify why we couldn't stream using waylandsink despite it supporting the UYVY format (according to gst-inspect-1.0 waylandsink).

[joanxie · Accepted solution]

the caps got from an inspect is just an template, waylandsink will query wayland compositor to find the right format list it can support.
Open waylandsink debug: export GST_DEBUG=waylandsink:7
you will got below kind of log:
waylandsink gstwaylandsink.c:569:gst_wayland_sink_get_caps:<waylandsink0> display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ RGB16, BGRx, RGBx, xBGR, ARGB, BGRA, RGBA, ABGR, xRGB, YUY2, NV12, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]

There are two different formats list, shm formats and dmabuf format

[kaartic_sn]

Hi joanxie‌,

Thanks for the useful information. Now I understand why waylandsink couldn't stream UYVY. It's clear from the debug information that it only supports YUY2 (YUYV) and not UYVY. Hope gst-inspect information was a little more precise. For the sake of completeness, I'm including what I get when I grep for the debug message using the following command:

$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! video/x-raw,width=$WIDTH,height=$HEIGHT ! waylandsink 2>&1 | grep gst_wayland_sink_get_caps
0:00:03.067151884   525      0x62e80a0 DEBUG            waylandsink gstwaylandsink.c:569:gst_wayland_sink_get_caps:<waylandsink0> display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ RGB16, BGRx, RGBx, xBGR, ARGB, BGRA, RGBA, ABGR, xRGB, YUY2, NV12, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:03.122352006   525      0x62e80a0 DEBUG            waylandsink gstwaylandsink.c:569:gst_wayland_sink_get_caps:<waylandsink0> display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ RGB16, BGRx, RGBx, xBGR, ARGB, BGRA, RGBA, ABGR, xRGB, YUY2, NV12, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:03.127376526   525      0x62e80a0 DEBUG            waylandsink gstwaylandsink.c:569:gst_wayland_sink_get_caps:<waylandsink0> display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ RGB16, BGRx, RGBx, xBGR, ARGB, BGRA, RGBA, ABGR, xRGB, YUY2, NV12, NV12_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]

Thanks,

Sivaraam