The problem of using v4l2dec and imxcompositor_g2d to accelerate imx8

[Hongtao]

Direct use

Spoiler

gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! v4l2jpegdec device=/dev/video2 ! waylandsink

The frame rate is only about 10.

Spoiler

gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! v4l2jpegdec device=/dev/video2 ! waylandsink sync=false

The frame rate is only about 30.

However, when two v4l2 elements are used together.gst-launch-1.0 fails.

Spoiler

gst-launch-1.0 v4l2src device=/dev/video2 io-mode=dmabuf ! image/jpeg,width=1280,height=720,framerate=30/1 ! v4l2jpegdec ! imxvideoconvert_g2d ! v4l2h264enc ! h264parse ! qtmux ! filesink location=test.mov -e

root@apalis-imx8:~# gst-launch-1.0 v4l2src device=/dev/video2  io-mode=dmabuf ! image/jpeg,width=1280,height=720,framerate=30/1 ! v4l2jpegdec ! imxvideoconvert_g2d ! v4l2h264enc ! h264parse ! qtmux ! filesink location=test.mov -e
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 3492.448173] [VPU Decoder] firmware version is 1.7.0
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
EOS on shutdown enabled -- waiting for EOS after Error
Waiting for EOS...
​

////////////////////////////////////////////////////////////////////////

同时imxcompositor_g2d配合v4l2dec也不能使用成功，比如下面

 gst-launch-1.0  imxcompositor_g2d  name=comp  sink_0::zorder=2 sink_0::alpha=0.7 sink_1::xpos=0 sink_1::ypos=0  sink_1::zorder=1 ! waylandsink sync=false sync=false  videotestsrc  !  comp.sink_0  v4l2src  device=/dev/video2  ! image/jpeg,width=1280,height=720,framerate=30/1  ! v4l2jpegdec !  comp.sink_1
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 3680.010757] [VPU Decoder] firmware version is 1.7.0

Although the pipeline is normal, the display is stuck.

Replacing v4l2jpegdec with jpegdec ，It can work normally, but the display will stuck every few seconds .

///////////////////////////

imxcompositor_g2d配合v4l2h264dec

    gst-launch-1.0  imxcompositor_g2d  name=comp  sink_0::zorder=2 sink_0::alpha=0.7 sink_1::xpos=0 sink_1::ypos=0  sink_1::zorder=1 ! waylandsink sync=false sync=false  videotestsrc  !  comp.sink_0  v4l2src  device=/dev/video4  ! video/x-h264,width=1280,height=720,framerate=30/1  ! v4l2h264dec !  comp.sink_1 ​

Same as above.



Hope to get a reply as soon as possible. Thank you very much

[joanxie]

what imx8 and bsp version do you use

[Hongtao]

你好。这个问题原因有找到吗？

[Hongtao]

root@apalis-imx8:~# uname -a
Linux apalis-imx8 5.4.91-5.2.0+git.6afb048a71e3 #1 SMP PREEMPT Wed Apr 7 08:36:44 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

[joanxie]

refer to the information you offered, I still couldn't tell what imx8 you use, imx8 series have imx8mq, imx8mm, imx8qm....., I don't know what imx8 you use, and it seems that you use the third party board and the third party software, nxp doesn't release 5.4.91-5.2.0 version bsp, so I couldn't reproduce this on nxp board

[Hongtao]

I saw the forum also has my similar question about v4l2jpegdec

[Hongtao]

I use toradex's imx8. Their technical support doesn't know how to solve it. Let me have a try here.

[joanxie]

as I mentioned before, first you need to know what imx8 you use, imx8 series has many products, for imx8mq, which doesn't support G2D, second, you used the third party software, I am not sure if they use the same gstreamer as nxp software does, but you can refer to the linux user guide maybe

"https://www.nxp.com/design/software/embedded-software/i-mx-software/embedded-linux-for-i-mx-applicat..."