Capturing YUV420 data over mipi csi

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Capturing YUV420 data over mipi csi

1,942 Views
efecan_icoz
Contributor III

Hi all,

I'm trying to capture camera data with strict latency limit to ~100ms. Currently i've connected a camera to my mipi csi bus (4 lane 786mhz) and streaming over standard yuv422/uyvy format. Since wayland doesn't support UYVY (gst-inspect says that it is supporting but in source UYVY doesn't supported in DMAbuf) as input i need to use imxvideoconvert_g2d and with that configuration i'm having ~160ms latency, command is below.

"gst-launch-1.0 v4l2src device="/dev/video0" ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)25/1" ! imxvideoconvert_g2d !  waylandsink"

I did not want to dive into wayland side to make it possible to stream uyvy, i want to switch yuv420 which i assume has the format NV12 and which i see every device (wayland + mipi port of imx8mm + my cam) supports it natively. Even though i found out that mx6s_capture driver (driver of mipi csi) doesn't cover the yuv420 implementation even the hw supports it.

I'll try to add the support of yuv420 to that driver but my initial trials have failed, is there any guide or anyone that has already done that before it would be nice to hear the details. Otherwise it's being quite hard to learn&understand the mipi driver.

Have a nice day,

Efe

Edit 1:

Currently i'm capturing image like below with this command:

`v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=NV12 --stream-mmap --stream-count=1 --stream-to=grab-nv12.raw`

pastedImage_2.png

And gstreamer with command below gives me this error:

v4l2src device="/dev/video0" ! "video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)25/1" !  waylandsink

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: Video memory can not be read from userspace.
Additional debug info:
../../../git/ext/wayland/gstwaylandsink.c(1127): gst_wayland_sink_show_frame (): /GstPipeline:pipeline0/GstWaylandSink:waylandsink0
Execution ended after 0:00:00.451851860
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

With the debug level 5 erroneous point is here:

0:00:00.862928777 2444 0x2e28ef70 DEBUG waylandsink wllinuxdmabuf.c:108:gst_wl_linux_dmabuf_construct_wl_buffer:<GstWlDisplay@0x2e36b810> Creating wl_buffer from DMABuf of size 3133440 (1920 x 1080), format NV12 modifier 0x0000000000000000
0:00:00.863007404 2444 0x2e28ef70 DEBUG GST_META gstmeta.c:88:gst_meta_api_type_register: register API "GstPhyMemMetaAPI"
0:00:00.863058654 2444 0x2e28ef70 DEBUG GST_META gstmeta.c:95:gst_meta_api_type_register: adding tag "video"
0:00:00.863079280 2444 0x2e28ef70 DEBUG GST_META gstmeta.c:95:gst_meta_api_type_register: adding tag "size"
0:00:00.863097655 2444 0x2e28ef70 DEBUG GST_META gstmeta.c:95:gst_meta_api_type_register: adding tag "orientation"
0:00:00.864253798 2444 0x2e28ef70 DEBUG waylandsink wllinuxdmabuf.c:175:gst_wl_linux_dmabuf_construct_wl_buffer:<ionallocator0> created linux_dmabuf wl_buffer (0xffff80003c80):1920x1080, fmt=NV12, 2 planes
0:00:00.864344675 2444 0x2e28ef70 DEBUG GST_PERFORMANCE gstminiobject.c:524:ensure_priv_data: allocating private data GstBuffer miniobject 0x2e36db40
0:00:00.864274673 2444 0x2e28ee80 DEBUG GST_POLL gstpoll.c:1414:gst_poll_wait: 0x2e28ee30: timeout :99:99:99.999999999
0:00:00.864486427 2444 0x2e28ef70 DEBUG fdmemory gstfdmemory.c:136:gst_fd_mem_map: 0xffff78024420: fd 29: mapped 0xffff85e9f000
0:00:00.864533678 2444 0x2e28ef70 DEBUG fdmemory gstfdmemory.c:136:gst_fd_mem_map: 0x2e1d46e0: fd 18: mapped 0xffff85ba7000
0:00:00.864605804 2444 0x2e28ef70 ERROR videometa gstvideometa.c:240:default_map: plane 1, no memory at offset 3110400
0:00:00.864628054 2444 0x2e28ef70 ERROR default video-frame.c:162:gst_video_frame_map_id: failed to map video frame plane 1
0:00:00.864666055 2444 0x2e28ef70 WARN waylandsink gstwaylandsink.c:1127:gst_wayland_sink_show_frame:<waylandsink0> error: Video memory can not be read from userspace.
0:00:00.864693180 2444 0x2e28ef70 DEBUG GST_MESSAGE gstelement.c:2110:gst_element_message_full_with_details:<waylandsink0> start
0:00:00.864761681 2444 0x2e28ef70 INFO GST_ERROR_SYSTEM gstelement.c:2141:gst_element_message_full_with_details:<waylandsink0> posting message: Video memory can not be read from userspace.
0 Kudos
4 Replies

1,284 Views
Jithin77
Contributor I

Hi @igorpadykov  , 

Have you resolved this memory issue ? If so can you please explain how can it be solved.

Thanks,

0 Kudos

1,778 Views
igorpadykov
NXP Employee
NXP Employee

Hi Efe

what processor used in the case, may be

useful to check sect.6.1 Capture Overview Linux Manual included in

Linux 5.4.3_1.0.0 Documentation

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos

1,778 Views
efecan_icoz
Contributor III

Hi I'm using iMX8M Mini as base board, i'll check the link in your message asap

0 Kudos

1,778 Views
igorpadykov
NXP Employee
NXP Employee

also may be useful

DART-MX8M GSTREAMER - Variscite Wiki 

Best regards
igor

0 Kudos