[iMX8MP][4K streaming] Failed to allocate required memory.

[khang_letruong]

Dear NXP Team,

I would like to know if there would be some specific configuration for being able to stream video from 4K sensor, please ? I am using the following command :

 GST_DEBUG=3 gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=YUY2,width=3840,height=2160,framerate=5/1" ! queue ! imxvideoconvert_g2d ! waylandsink > gst-debug-4k.txt 2>&1

and struggling with the following error :

Setting pipeline to PAUSED ...
0:00:00.494213500   974 0xaaaaced3ad20 WARN                    v4l2 v4l2_calls.c:530:gst_v4l2_subscribe_event:<v4l2src0> /dev/video0 cannot subscribe V4L2_EVENT_SOURCE_CHANGE event.
0:00:00.494305000   974 0xaaaaced3ad20 WARN                    v4l2 v4l2_calls.c:538:gst_v4l2_subscribe_event:<v4l2src0> /dev/video0 cannot subscribe V4L2_EVENT_EOS event.
0:00:00.494339625   974 0xaaaaced3ad20 WARN                    v4l2 v4l2_calls.c:545:gst_v4l2_subscribe_event:<v4l2src0> /dev/video0 cannot subscribe V4L2_EVENT_DECODE_ERROR event.
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1
0:00:06.306039000   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1
0:00:06.318713500   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d0.GstPad:src: caps = video/x-raw, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1, format=(string)BGRx
0:00:06.318993750   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
0:00:06.319405125   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = video/x-raw, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1, format=(string)BGRx
0:00:06.320007375   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
0:00:06.320084375   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
0:00:06.320202000   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
0:00:06.320372500   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
0:00:06.320476125   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
0:00:06.320547125   974 0xaaaacee1b120 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 6 for RGB format, using RGB
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)3840, height=(int)2160, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:6:11:1
0:00:07.020096875   974 0xaaaacee1ab00 ERROR          v4l2allocator gstv4l2allocator.c:740:gst_v4l2_allocator_start:<v4l2src0:pool0:src:allocator> error requesting 3 buffers: Resource temporarily unavailable
0:00:07.020211125   974 0xaaaacee1ab00 ERROR         v4l2bufferpool gstv4l2bufferpool.c:926:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> we received 0 buffer from device '/dev/video0', we want at least 2
0:00:07.020254500   974 0xaaaacee1ab00 ERROR             bufferpool gstbufferpool.c:559:gst_buffer_pool_set_active:<v4l2src0:pool0:src> start failed
0:00:07.020296625   974 0xaaaacee1ab00 WARN                 v4l2src gstv4l2src.c:659:gst_v4l2src_decide_allocation:<v4l2src0> error: Failed to allocate required memory.
0:00:07.020327500   974 0xaaaacee1ab00 WARN                 v4l2src gstv4l2src.c:659:gst_v4l2src_decide_allocation:<v4l2src0> error: Buffer pool activation failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
../git/sys/v4l2/gstv4l2src.c(659): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
0:00:07.020575750   974 0xaaaacee1ab00 WARN                 basesrc gstbasesrc.c:3347:gst_base_src_prepare_allocation:<v4l2src0> Subclass failed to decide allocation
Execution ended after 0:00:06.524860500
Setting pipeline to NULL ...
0:00:07.020640125   974 0xaaaacee1ab00 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:07.020678375   974 0xaaaacee1ab00 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

Meanwhile, it works well with my 1440x1080 sensor (just by switching the dtb) :

 GST_DEBUG=3 gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=YUY2,width=1440,height=1080" ! queue ! imxvideoconvert_g2d ! waylandsink > gst-debug-1440x1080.txt 2>&1

Setting pipeline to PAUSED ...
0:00:00.498230625   991 0xaaaae2a90690 WARN                    v4l2 v4l2_calls.c:530:gst_v4l2_subscribe_event:<v4l2src0> /dev/video0 cannot subscribe V4L2_EVENT_SOURCE_CHANGE event.
0:00:00.498305750   991 0xaaaae2a90690 WARN                    v4l2 v4l2_calls.c:538:gst_v4l2_subscribe_event:<v4l2src0> /dev/video0 cannot subscribe V4L2_EVENT_EOS event.
0:00:00.498336375   991 0xaaaae2a90690 WARN                    v4l2 v4l2_calls.c:545:gst_v4l2_subscribe_event:<v4l2src0> /dev/video0 cannot subscribe V4L2_EVENT_DECODE_ERROR event.
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:00:00.574127750   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d0.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)BGRx
0:00:00.574332500   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.574563125   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.574951375   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)BGRx
0:00:00.575204500   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.575268250   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.575555375   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.575705750   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.575749125   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
0:00:00.575817250   991 0xaaaae2beb520 WARN              video-info video-info.c:526:gst_video_info_from_caps: invalid matrix 3 for RGB format, using RGB
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1440, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:00:00.920576250   991 0xaaaae2beb520 WARN             waylandsink wlwindow.c:441:gst_wl_window_new_internal: init surface_state fail, fallback to scale=1 fullscreen (1920x1048)
handling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:13.554667000
Setting pipeline to NULL ...
Total showed frames (395), playing for (0:00:13.554668125), fps (29.141).

CMA allocation :

root@imx8mpevk:~# dmesg | grep -i cma
[    0.000000] Reserved memory: created CMA memory pool at 0x00000000c4000000, size 960 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Memory: 4759752K/6258688K available (18304K kernel code, 1914K rwdata, 7524K rodata, 3648K init, 550K bss, 515896K reserved, 983040K cma-reserved)

I am using BSP LF-5.10.72-2.2.0. Attached are the output of gst-launch command and associated dmesg in both cases.

Thanks in advance and best regards,

Khang

[malik_cisse]

Hi Khang,

I can see that you have also had CMA allocation issues and maybe you have some thoughts about my current issue.

I am using this Gstreamer pipeline in QT5. EGLFS platform plugin is used (no Wayland/weston):
descr = g_strdup_printf ("v4l2src name=video_source device=/dev/v4l/by-path/platform-vvcam-video.0-video-index0 ! video/x-raw, width=3840,height=2160 ! "
"videocrop name=video_crop ! imxvideoconvert_g2d ! interpipesink name=src_1 sync=false ");

I am dynamically zooming by setting videocrop parameters (left_crop, right_crop etc.) 

I eventually get this type of CMA related error and video stops playing:

[10337.440327] alloc_contig_range: [6d900, 6da77) PFNs busy

When searching for "alloc_contig_range:  PFNs busy" issue, one finds it is related to CMA. here we find some more details about CMA:

https://developer.toradex.com/linux-bsp/how-to/linux-features/contiguous-memory-allocator-cma-linux/

Further investigation show the actual issue seems to be located in imxvideoconvert_g2d (G2D GPU Accelerator) element.

When I check this command, videocrop plug-in doesn't crop image buffer itself. It's just set crop metadata and pass this information to imxvideoconvert_g2d. And imxvideoconvert_g2d plug-in check crop metadata about input image buffer and if it's not null, it crop image buffer.

The reason why videocrop plug-in doesn't crop image itself, it's running as in-place mode.

Thus, something must still be buggy in imxvideoconvert_g2d but I could not figure out what it is. Checking source code and fixes did not help either:
https://source.codeaurora.org/external/imx/imx-gst1.0-plugin/tree/plugins/videoconvert/gstimxvideoco...

Thx,

[khang_letruong]

Hi @malik_cisse ,

I would say that the alloc_contig_range: [6d900, 6da77) PFNs busy relates to CMA allocation failure. 

Could you check dmesg | grep -i cma ?

Regards,
Khang

[malik_cisse]

root@amrum-fumu1-imx8mp-2:~# dmesg | grep -i cma 
[ 0.000000] Reserved memory: created CMA memory pool at 0x0000000056000000, size 960 MiB
[ 0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[ 0.000000] Memory: 732784K/1794048K available (18428K kernel code, 2370K rwdata, 7768K rodata, 6400K init, 2970K bss, 78224K reserved, 983040K cma-reserved)
[ 258.968075] cma_release+0x6c/0x190
[ 258.968079] ion_cma_free+0x30/0x50

result is like what you had.

There is enough CMA allocated and when the issue occurs, I can see that plenty of free CMA space is available.

[khang_letruong · Accepted solution]

The problem is resolved : The sensor mode in isp-imx must match the index among supported modes in lower-level isp-vvcam driver.

Regards,

Khang