Using Gstreamer vpudec on imx8m for Mjpeg and H264

[andy_bern]

Hi, i would like to use hardware acceleration to decode an udp rtp Mjpeg or H264 stream (Mjpeg prefered) with Gstreamer on the imx8m (quad). Im using an Yocto (sumo) BSP. The Display should be the Mipi Dsi (there is an Mipi Dsi to Lvds video converter). On this BSP there is an xwayland (weston) running so for Gstreamer waylandsink can be used. Weston can also be stopped so kmssink can then be used.

The goal is only to use the gpu (vpu) for decoding, converting, ... so the cpu resources are free.

Questions:

1) How would a gstreamer pipeline look with vpudec and waylandsink or kmssink for mjpeg or h264?

2) Is fbdevsink slower then kmssink or waylandsink?

3) There is no matching colorspace between the output of vpudec and the input of fbdevsink. Is there a gstreamer plugin that uses hardware accelerated colorspace conversion on the imx8m (maybe imxvideoconvert_ipu)?

4) Does wayland use the gpu to some degree (some percent), so that kmssink would be faster then waylandsink because it doesn't have to render the "desktop environment"?

[benson_lin]

Dera andy_bern

Regarding your question 1, I have some questions!

Are these commands useful? ( gst-launch-1.0 you_src  ! vpudec ! queue ! videoconvert ! waylandsink )

Share some information for you:

My platform is imx8mqevk, I have a camera, the camera supports mjpeg format, I directly output to HDMI, the result is very low frame rate and latency. ( My test resolution is 1280x720 )

I am curious about your test results. Is the frame rate also very low? Does it feel lag?

Thanks.

[Bio_TICFSL]

Hello Andy,

1) for MPEG4 hardware accelerated decode it should be:

gst-launch-1.0 filesrc location=<filename.mp4> ! \
qtdemux name=d d.video_0 ! queue ! mpeg4videoparse ! v4l2mpeg4dec ! \
imxvideoconvert_g2d ! queue ! waylandsink

for MPEG2 HW accelerated decode:

gst-launch-1.0 filesrc location=<filename.ts> ! \
qtdemux name=d d.video_0 ! queue ! mpegvideoparse ! v4l2mpeg2dec ! \
imxvideoconvert_g2d ! queue ! waylandsink

2) kmmsink is a video sink using linux kernel mode setting API while wayland sink is output on wayland interface. fbdevsink is not supported by MX8 that the reason of being slower.

3) there is no matching for fbdevsink.

4) Yes, for using the kmmsink pluggins, one need to stop weston interface before: $ systemctl stop weston.

Regards