Question about imxvideoconvert_g2d plug-in for imx8qm

[dongjin]

Hi all,

I'm using imx8qm and testing gstreamer.

When I check gstimxvideoconvert.c, 'GST_IMX_VIDEO_VIDEOCROP_META_DEFAULT' is false.

So I couldn't set crop property with imxvideoconvert_g2d plug-in.

Can I set 'GST_IMX_VIDEO_VIDEOCROP_META_DEFAULT' to true ?

Could you let me know why it's false as default? Any h/w limitation?

Thanks,

Dongjin Ha.

[malik_cisse]

maybe One can help with this question related to the topic discussed here.

 The device is imx8mp and QT-EGLFS is used (no Wayland/weston)

When using this pipeline

v4l2src device=/dev/video0 ! video/x-raw, width=3840,height=2160 ! videocrop name=video_crop ! imxvideoconvert_g2d ! video/x-raw, width=480, height=800 ! autovideosink 

and dynamically setting videocrop parameters (left_crop, right_crop etc.)

One get following message on the debug console

[10124.322313] ------------[ cut here ]------------
[10124.326970] 375 pages are still in use!
[10124.330935] WARNING: CPU: 2 PID: 1664 at mm/page_alloc.c:8525 free_contig_range+0xa4/0xb8
[10124.339112] Modules linked in: vvcam_isp(O) vvcam_dwe(O) vvcam_video(O) imx8_media_dev(C) imx477(O) crct10dif_ce lrdmwl_pcie(O) lrdmwl(O) asix usbnet mac80211(O) g_mass_stora]
[10124.358868] CPU: 2 PID: 1664 Comm: QSGRenderThread Tainted: G        WC O      5.4.70-bsp-yocto-fsl-i.mx8mp-pd21.1.1 #1
[10124.369648] Hardware name: PHYTEC phyBOARD-Pollux i.MX8MP (DT)
[10124.375481] pstate: 40000005 (nZcv daif -PAN -UAO)
[10124.380273] pc : free_contig_range+0xa4/0xb8
[10124.384541] lr : free_contig_range+0xa4/0xb8
[10124.388807] sp : ffff80001296bbe0
[10124.392119] x29: ffff80001296bbe0 x28: ffff0000751c0000 
[10124.397430] x27: 0000000000000000 x26: 0000000000000000 
[10124.402742] x25: ffff000079fb3840 x24: ffff000079ee6e40 
[10124.408052] x23: 0000000000000001 x22: ffff8000118facd0 
[10124.413362] x21: 0000000001991dc0 x20: 0000000000000177 
[10124.418673] x19: 0000000001991dc0 x18: 0000000000000000 
[10124.423983] x17: 0000000000000000 x16: 0000000000000000 
[10124.429293] x15: ffff00007a53a330 x14: ffffffffffffffff 
[10124.434603] x13: 0000000000086a30 x12: ffff8000122c1000 
[10124.439914] x11: ffff800012092000 x10: 00000000000019c0 
[10124.445224] x9 : ffff80001296b8c0 x8 : ffff0000751c1a20 
[10124.450534] x7 : 00000000000e7ef0 x6 : 0000000000000002 
[10124.455845] x5 : ffff00007ddbe180 x4 : 0000000000000001 
[10124.461155] x3 : ffff00007ddbe180 x2 : 0000000000000007 
[10124.466466] x1 : 652b72273a846500 x0 : 0000000000000000 
[10124.471777] Call trace:
[10124.474225]  free_contig_range+0xa4/0xb8
[10124.478148]  cma_release+0x6c/0x190
[10124.481637]  ion_cma_free+0x30/0x50
[10124.485125]  ion_buffer_destroy+0x5c/0xd0
[10124.489134]  _ion_buffer_destroy+0x40/0x50
[10124.493229]  ion_dma_buf_release+0x20/0x48
[10124.497325]  dma_buf_release+0x54/0xc8
[10124.501073]  __dentry_kill+0x130/0x1c0
[10124.504820]  dput+0x1c0/0x310
[10124.507786]  __fput+0xd0/0x230
[10124.510839]  ____fput+0x20/0x30
[10124.513981]  task_work_run+0x88/0x138
[10124.517644]  do_notify_resume+0x208/0x6a0
[10124.521652]  work_pending+0x8/0x10
[10124.525052] ---[ end trace 67d40e2238d7e09e ]---

despite this message video may play fine but subsequent press on zoom will eventually freeze display and one get this message:

[10337.440327] alloc_contig_range: [6d900, 6da77) PFNs busy
[10337.446533] alloc_contig_range: [6da00, 6db77) PFNs busy

When searching for "alloc_contig_range:  PFNs busy" issue, one finds it is related to CMA. here we find some more details about CMA:

https://developer.toradex.com/linux-bsp/how-to/linux-features/contiguous-memory-allocator-cma-linux/

Trying to get rid of CMA using techniques describe bellow did not succeed as none of the points really apply to our case:

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/How-to-get-rid-of-CMA/ta-p/1123287

Also we are allocating enough memory for CMA in the device tree:

root@amrum-fumu1-imx8mp-2:~# dmesg |grep CMA
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000056000000, size 960 MiB

When monitoring CMA usage before and after the error occurs one can see that the amount of memory does not drop dramatically. we are staying around 700MB of headroom, so lack of memory is not the issue:

root@amrum-fumu1-imx8mp-2:~# cat /proc/meminfo | grep -i cma
CmaTotal:         983040 kB
CmaFree:          715624 kB

Further investigation show the actual issue seems to be located in imxvideoconvert_g2d (G2D GPU Accelerator) element.
The proof is given when replacing imxvideoconvert_g2d with videoconvert. The issue is gone but video does not play smoothly lacking of HW acceleration:

descr = g_strdup_printf ("v4l2src name=video_source device=/dev/v4l/by-path/platform-vvcam-video.0-video-index0 ! video/x-raw, width=3840,height=2160 ! "

                         "videocrop name=video_crop ! videoconvert ! interpipesink name=src_1 sync=false ");

Some issues related to videocrop have been fixed in newer releases of imxvideoconvert_g2d as can be seen here:
https://source.codeaurora.org/external/imx/imx-gst1.0-plugin/log/plugins/videoconvert/gstimxvideocon...

Does this sound familiar to you and how do you think this can be solved?

[dongjin]

I think your gstreamer pipeline set crop parameters at imxvideoconvert_g2d.

I guess you can set it after videocrop plug-in and it'll be better than yours.

v4l2src device=/dev/video0 ! video/x-raw, width=3840,height=2160 ! videocrop name=video_crop ! imxvideoconvert_g2d ! video/x-raw, width=480, height=800 ! autovideosink 

[b36401]

You can refer to VideoCropMeta structure:
https://gstreamer.pages.freedesktop.org/gstreamer-rs/stable/latest/docs/gstreamer_video/video_meta/s...

[dongjin]

Dear NXP,

I checked that you said about videocropmeta, but I couldn't find a way that I want.

Can I set crop property with imxvideoconvert_g2d plug-in? I want to crop image with h/w acceleration(dpu).

Thanks,

Dongjin Ha.

[quercuspau]

Hello

I dont understand the answer. I rephrase the question: Is it possible to crop images using imxvideoconvert_g2d? If yes, could you share a gstreamer pipeline example?

BR

[dongjin]

Hi @quercuspau 

You can use below command for crop image by imxvideoconvert_g2d plug-in.

gst-launch-1.0 v4l2src device=/dev/video1 ! videocrop left=300 ! imxvideoconvert_g2d ! autovideosink

When I check this command, videocrop plug-in doesn't crop image buffer itself. It's just set crop metadata and pass this information to imxvideoconvert_g2d. And imxvideoconvert_g2d plug-in check crop metadata about input image buffer and if it's not null, it crop image buffer.

The reason why videocrop plug-in doesn't crop image itself, it's running as in-place mode. If it's running as normal mode, it will crop image buffer itself and it's not we expected.

I'm checking this pipeline with imx8qm, so I'm not sure it can be running on your side.

Thanks,

Dongjin Ha.

[quercuspau]

Hi,

Thanks for your help it works

[quercuspau]

I have the same question for imx8mp. How to crop using imxvideoconvert_g2d plug-in?

BR