gstreamer's stream is sliding on iMX8M Mini

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

gstreamer's stream is sliding on iMX8M Mini

Jump to solution
1,402 Views
efecan_icoz
Contributor III

Hi all,

I'm having bizarre problem with my imx8mm board, i've connected my camera over MIPI CSI2 bus 4 lane mode and my camera sends its frames with UYVY format. I'm using yocto warrior with 4.19.35 kernel. Whenever i capture single raw frame i get perfect result. Like below:
pastedImage_1.png

But whenever i start stream with below command i get frames sliding down vertically like below:

gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)25/1" ! imxvideoconvert_g2d !  waylandsink‍

pastedImage_1.png

I've

- Tried using "videoconvert" but it streams with 0.8 fps and problem still exists in that configuration too.

- Tried _not_ using any videoconvert but gstreamer gives not-negotiated error like below:

0:00:00.376810459 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:944:gst_bus_add_watch_full_unlocked:<bus1> New source 0x3dd60340 with id 15
0:00:00.376825334 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1220:gst_bus_poll: running mainloop 0x3dd5ffd0
0:00:00.376853084 2688 0x3dcaeb70 DEBUG GST_PERFORMANCE gstminiobject.c:428:gst_mini_object_make_writable: copy GstCaps miniobject 0x3dcae8f0 -> 0x3dcaef20
0:00:00.376862710 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:544:gst_bus_timed_pop_filtered:<bus1> got message 0x3dd7b3b0, state-changed from capsfilter0, type mask is 4294967295
0:00:00.376903335 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:837:gst_bus_source_dispatch:<bus1> source 0x3dd60340 calling dispatch with state-changed message: 0x3dd7b3b0, time 99:99:99.999999999, seq-num 15, element 'capsfilter0', GstMessageStateChanged, old-state=(GstState)GST_STATE_READY, new-state=(GstState)GST_STATE_PAUSED, pending-state=(GstState)GST_STATE_VOID_PENDING;
0:00:00.376930586 2688 0x3dcaeb70 DEBUG waylandsink gstwaylandsink.c:597:gst_wayland_sink_get_caps:<waylandsink0> display caps: video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16, YUY2, NV12, I420 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.376946586 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1107:poll_func: mainloop 0x3dd5ffd0 quit
0:00:00.376957086 2688 0x3dcaeb70 DEBUG GST_PADS gstpad.c:4076:gst_pad_query:<waylandsink0:sink> sent query 0x3dcaeed0 (caps), result 1
0:00:00.376969586 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:842:gst_bus_source_dispatch:<bus1> source 0x3dd60340 handler returns 1
0:00:00.376984211 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1222:gst_bus_poll: mainloop stopped 0x3dd5ffd0
0:00:00.376998086 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1464:gst_bus_remove_signal_watch:<bus1> removing signal watch 15
0:00:00.377009337 2688 0x3dcaeb70 DEBUG GST_CAPS gstutils.c:3111:gst_pad_peer_query_caps:<capsfilter0:src> peer query returned video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16, YUY2, NV12, I420 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.377013712 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:863:gst_bus_source_finalize:<bus1> finalize source 0x3dd60340
0:00:00.377038337 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1235:gst_bus_poll:<bus1> finished poll with message 0x3dd7b3b0
0:00:00.377057087 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:684:gst_base_transform_query_caps:<capsfilter0:sink> peer caps video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16, YUY2, NV12, I420 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.377072963 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:687:gst_base_transform_query_caps:<capsfilter0:sink> our template ANY
0:00:00.377078963 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:944:gst_bus_add_watch_full_unlocked:<bus1> New source 0x3dd60340 with id 17
0:00:00.377091588 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1220:gst_bus_poll: running mainloop 0x3dc24740
0:00:00.377112213 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:689:gst_base_transform_query_caps:<capsfilter0:sink> intersected video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16, YUY2, NV12, I420 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.377127463 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:544:gst_bus_timed_pop_filtered:<bus1> got message 0x3dd7b430, stream-status from src, type mask is 4294967295
0:00:00.377128713 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:471:gst_base_transform_transform_caps:<capsfilter0> transform caps (direction = 1)
0:00:00.377224090 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:837:gst_bus_source_dispatch:<bus1> source 0x3dd60340 calling dispatch with stream-status message: 0x3dd7b430, time 99:99:99.999999999, seq-num 18, element 'src', GstMessageStreamStatus, type=(GstStreamStatusType)GST_STREAM_STATUS_TYPE_CREATE, owner=(GstElement)"\(GstV4l2Src\)\ v4l2src0", object=(GstTask)"\(GstTask\)\ v4l2src0:src";
0:00:00.377230840 2688 0x3dcaeb70 DEBUG capsfilter gstcapsfilter.c:296:gst_capsfilter_transform_caps:<capsfilter0> input: video/x-raw, format=(string){ BGRA, BGRx, RGB16, I420, NV12, YUY2 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw(memory:DMABuf), format=(string){ BGRA, BGRx, RGB16, YUY2, NV12, I420 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.377265340 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1107:poll_func: mainloop 0x3dc24740 quit
0:00:00.377266090 2688 0x3dcaeb70 DEBUG capsfilter gstcapsfilter.c:297:gst_capsfilter_transform_caps:<capsfilter0> filter: (NULL)
0:00:00.377291216 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:842:gst_bus_source_dispatch:<bus1> source 0x3dd60340 handler returns 1
0:00:00.377304091 2688 0x3dcaeb70 DEBUG capsfilter gstcapsfilter.c:299:gst_capsfilter_transform_caps:<capsfilter0> caps filter: video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)25/1
0:00:00.377305966 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1222:gst_bus_poll: mainloop stopped 0x3dc24740
0:00:00.377329716 2688 0x3dcaeb70 DEBUG capsfilter gstcapsfilter.c:300:gst_capsfilter_transform_caps:<capsfilter0> intersect: EMPTY
0:00:00.377347967 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1464:gst_bus_remove_signal_watch:<bus1> removing signal watch 17
0:00:00.377356092 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:498:gst_base_transform_transform_caps:<capsfilter0> to: EMPTY
0:00:00.377364092 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:863:gst_bus_source_finalize:<bus1> finalize source 0x3dd60340
0:00:00.377369967 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:697:gst_base_transform_query_caps:<capsfilter0:sink> transformed EMPTY
0:00:00.377383842 2688 0x3dcaeb70 DEBUG basetransform gstbasetransform.c:731:gst_base_transform_query_caps:<capsfilter0> returning EMPTY
0:00:00.377384217 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1235:gst_bus_poll:<bus1> finished poll with message 0x3dd7b430
0:00:00.377420843 2688 0x3dcaeb70 DEBUG GST_PADS gstpad.c:4076:gst_pad_query:<capsfilter0:sink> sent query 0x3dcaee80 (caps), result 1
0:00:00.377445468 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:944:gst_bus_add_watch_full_unlocked:<bus1> New source 0x3dd60340 with id 19
0:00:00.377449843 2688 0x3dcaeb70 DEBUG GST_CAPS gstutils.c:3111:gst_pad_peer_query_caps:<v4l2src0:src> peer query returned EMPTY
0:00:00.377464968 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:1220:gst_bus_poll: running mainloop 0x3dd5ffd0
0:00:00.377477218 2688 0x3dcaeb70 DEBUG v4l2src gstv4l2src.c:524:gst_v4l2src_negotiate:<v4l2src0> caps of peer: EMPTY
0:00:00.377493844 2688 0x3dcaeb70 DEBUG v4l2src gstv4l2src.c:530:gst_v4l2src_negotiate:<v4l2src0> intersect: EMPTY
0:00:00.377497844 2688 0x3dbeca00 DEBUG GST_BUS gstbus.c:544:gst_bus_timed_pop_filtered:<bus1> got message 0x3dd7b4b0, state-changed from v4l2src0, type mask is 4294967295
0:00:00.377509844 2688 0x3dcaeb70 DEBUG basesrc gstbasesrc.c:3009:gst_base_src_loop:<v4l2src0> Not negotiated
0:00:00.377525844 2688 0x3dcaeb70 DEBUG basesrc gstbasesrc.c:3025:gst_base_src_loop:<v4l2src0> pausing task, reason not-negotiated
0:00:00.377537844 2688 0x3dcaeb70 DEBUG GST_PADS gstpad.c:6205:gst_pad_pause_task:<v4l2src0:src> pause task

I don't understand that even wayland supports UYVY format why DMAbuf doesn't support it.

Also i couldn't find why my stream is sliding down, it's obviousy has vsync because i can capture 1 frame without any glitch.

Also as side note i can't connect CSI interrupt signal from camera, could this be the problem ?

Have a nice day

0 Kudos
1 Solution
1,275 Views
efecan_icoz
Contributor III

Reason of this upward or downward sliding is happening due to incorrect "vertical spacing", when this field set to correct value sliding will be stopped.

Also I don't know the internals in detail but waylandsink doesn't support for UYVY but YUYV(YUY2), that requires the usage imxvideoconvert. Details of waylandsink could be found in gst-plugins-bad repository.

View solution in original post

0 Kudos
3 Replies
1,276 Views
efecan_icoz
Contributor III

Reason of this upward or downward sliding is happening due to incorrect "vertical spacing", when this field set to correct value sliding will be stopped.

Also I don't know the internals in detail but waylandsink doesn't support for UYVY but YUYV(YUY2), that requires the usage imxvideoconvert. Details of waylandsink could be found in gst-plugins-bad repository.

0 Kudos
1,275 Views
joanxie
NXP TechSupport
NXP TechSupport

it seems that waylandsink supports YUYV,but not UYVY, you can refer to my another topic:

https://community.nxp.com/message/1146729?commentID=1146729#comment-1146729 

0 Kudos
1,275 Views
efecan_icoz
Contributor III

Yes after digging the source of waylandsink in "gstreamer-plugins-bad" repo i've also found out that DMAbuffered stream of waylandsink doesn't support UYVY.

Now i'm having latency issues with imxvideoconvert_g2d converting UYVY to suitable format for waylandsink. I'm having around 160-200ms latency, i wonder where can i find the source of the imxvideoconvert_g2d or can i make it faster a little bit.

0 Kudos