i.MX8QM MEK 2160p60 MP4 playback

[stephanhoffmann]

Hello all,

I am evaluating the multimedia capabilities of the iMX8QM using the MEK board with the Yocto Image NXP i.MX Release Distro 5.15-kirkstone, Linux imx8qmmek 5.15.71+g3313732e9984. I have a screen connected to the on board HDMI connector and the results are almost the same weather I use a HD or a 4k screen.

Currently I am struggling with 4k video. I am using the Big Buck Bummy files with "normal" in the name from https://download.blender.org/demo/movies/BBB/

All of them up to and including 2180p30 play well with this gstreamer pipeline:

gst-launch-1.0 filesrc location=/media/card/bbb_sunflower_2160p_30fps_normal.mp4 ! decodebin ! imxvideoconvert_g2d ! waylandsink

2160p60 plays like a slide show and there are lots of the following messages while the CPU load is low, gst-launch about 15%:

WARNING: from element /GstPipeline:pipeline0/GstWaylandSink:waylandsink0: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3143): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstWaylandSink:waylandsink0:
There may be a timestamping problem, or this computer is too slow.

I read somewhere that this might be related to the CMA configuration, but as soon as I try to change this (either using the cma=xxx kernel command line option or modifying the device tree) I get the kernel message

[ 0.000000] OF: reserved mem: node linux,cma compatible matching fail

and video playback does not work at all.

Can anybody confirm that the iMX8QM is capable to play this video and tell me how that can be achieved?

Thank you very much.

[Dhruvit]

Hi @stephanhoffmann,

I hope you are doing well

i.MX8QM has the features as below.

Display Controller->

• Supports single UltraHD 4Kp60 display or up to 4 independent FullHD 1080p60 displays
• Up to 18-layer composition
• Complementary 2D blitting engines and online warping functionality
• Integrated Failover Path (SafeAssure) to ensure display content stays valid even in event of a software failure

One can check about this in RM in the display section.
https://www.nxp.com/webapp/Download?colCode=IMX8QMRM

Please try with the below kernel command line argument

• cma=r=20M cma_map=video,camera=r

The first instructs CMA to allocate a region of 20 MiB and use the first available memory allocator on it.  The second is that drivers named "video" and "camera" are to be granted memory from the previously defined region.
   
Please share the observation.

Thanks & Regards,
Dhruvit Vasavada

[stephanhoffmann]

Hello Dhruvit Vasavada,

thank you for your advice. I tried your suggested kernel command line parameters but with no success:

1. I get this message on boot:

[ 0.000000] Kernel command line: console=ttyLP0,115200 earlycon root=/dev/mmcblk1p2 rootwait rw video=mxcfb0:dev=hdmi,1920x1200M@60,if=RGB24 cpufreq.default_governor=SCHEDUTIL cma=r=20M cma_map=video,camera=r
[ 0.000000] Unknown kernel command line parameters "cma_map=video,camera=r", will be passed to user space.
[ 0.000000] Memory: 5527280K/6092800K available (18560K kernel code, 1548K rwdata, 7172K rodata, 2944K init, 543K bss, 565520K reserved, 0K cma-reserved)

With the normal command line I get

[ 0.000000] Kernel command line: console=ttyLP0,115200 earlycon root=/dev/mmcblk1p2 rootwait rw video=mxcfb0:dev=hdmi,1920x1200M@60,if=RGB24 cpufreq.default_governor=SCHEDUTIL
...
[ 0.000000] Memory: 4544240K/6092800K available (18560K kernel code, 1548K rwdata, 7172K rodata, 2944K init, 543K bss, 565520K reserved, 983040K cma-reserved)

2. Gstreamer does not run any more:

# gst-launch-1.0 filesrc location=/media/card/bbb_sunflower_1080p_30fps_normal.mp4 ! decodebin ! imxvideoconvert_g2d ! waylandsink 
g2d_dev_open: open /dev/dma_heap/linux,cma-uncached failed, No such file or directory

Kind regards

Stephan

[Dhruvit]

Hi @stephanhoffmann,

I hope you are doing well.

Please make sure all the elements mentioned in the gst-launch are built into the imx platform.
$ gst-launch-1.0 filesrc location=~/Desktop/tuhi.mp4 ! decodebin name=dec ! videoconvert ! autovideosink dec. ! audioconvert ! audioresample ! alsasink

One can also try the below commands to playback 4K video using the playbin element.
$ gst-launch-1.0 playbin uri=file:<location_of_file>.mp4

$ gst-launch-1.0 filesrc location=<file.mp4> ! qtdemux ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! 'video
xraw(memory:NVMM),format=NV12,width=1920,height=1080' ! queue ! nvegltransform ! fpsdisplaysink text-overlay=0 video-sink=nveglglessink sync=false -v

Please make sure the screen which is being used is in 4k60p mode
$ cat /sys/class/graphics/fb0/mode

Please refer the chapter-11 of imx8gstreameruserguide.
https://community.nxp.com/pwmxy87654/attachments/pwmxy87654/imx-processors%40tkb/15/2/i.MX8GStreamer...

Thanks & Regards,
Dhruvit Vasavada

[stephanhoffmann]

Hello Dhruvit,

could you please be so kind and take an imx8qmmek, flash

ID=fsl-imx-wayland
NAME="NXP i.MX Release Distro"
VERSION="5.15-kirkstone (kirkstone)"
VERSION_ID=5.15-kirkstone
PRETTY_NAME="NXP i.MX Release Distro 5.15-kirkstone (kirkstone)"
DISTRO_CODENAME="kirkstone"

 onto it, download Big Buck Bunny from blender.org and try to play it as described in the i.MX 8 GStreamer User Guide paragraph 3.2.1?

gst-launch-1.0 filesrc location=/run/media/mmcblk0p1/bbb_sunflower_2160p_60fps_normal.mp4 ! qtdemux name=d d.video_0 ! queue ! h264parse ! v4l2h264dec  ! imxvideoconvert_g2d ! queue ! waylandsink

The result that I see is almost the same with a 1080 or a 4k display.

Thank you very much

Stephan

[Dhruvit]

Hi @stephanhoffmann,

I hope you are doing well.

Please share the display part number you are using, need to check whether it is 4k60p supported or not.

Thanks & Regards,
Dhruvit Vasavada

[stephanhoffmann]

Hello Dhruvit,

I am expecting the same results with the following display configurations:

* A Samsung SyncMaster 2443 24" 1920x1200p60 HDMI screen

* A Samsung SyncMaster 2443NW 24" 1920x1200p60 analogue screen attached with a HDMI to VGA adapter

* The above screens attached with a 4k to HD HDMI downscaler

All three configurations give roughly the same performance.

That said the video playback issues are not related to the display.

Kind regards

Stephan

[Dhruvit]

Hi @stephanhoffmann,

I hope you are doing well.

If the display is 4K supportive then this is not a display-related issue.
* The above screens are attached with a 4k to HD HDMI downscaler
->If this connection is made the definitely the output will be converted into 1080p and one cant has 4k output at hdmi interface connected screen.
->4K HDMI Down Scaler can convert 4K signal resolution to 1080p in real-time with no latency.
->So if one is using this and getting 1080p output then that explains, which is expected behavior.

Thanks & Regards,
Dhruvit Vasavada

[stephanhoffmann · Accepted solution]

Hello Dhruvit,

thank you for your investigtion. Finally I found the solution myself:

According to i.MX 8QuadMax Industrial Applications Processors Data Sheet Table 1 the processor simply cannot decode h264 with 4kp60 but only with 4kp30 max.

[stephanhoffmann]

Hello Dhruvit,

thank you for your response. Unfortunatelly it does not seem to help much.

I do not know how to tell wether I have the right pipeline elements. I thought when using your officiel image that should be the case.

When using your first suggestion (videoconvert instead of imxvideoconvert_g2d) the performance is very poor, what I expected.

Also playbin gives very poor performance even with a 1080p file.

Your third suggestion does not work because I do not have any Gstreamer element starting with nvv.

My /sys/class/graphics/fb0/mode does not contain anything, but I am definitively at 4k. Did I mention that I am running weston?

# dmesg | grep Mode
[ 6.951881] [drm] Mode: 3840x2160p594000

chapter-11 of imx8gstreameruserguide refers to multiple displays, but I have only one attached to the native HDMI port.

To be on the safe side I tested the pipeline mentioned in iMX Gstreamer Guide 3.2.1. H.264 Decode (HW Accelerated Decode)

gst-launch-1.0 filesrc location=<filename.mp4> ! \
qtdemux name=d d.video_0 ! queue ! h264parse ! v4l2h264dec ! \
imxvideoconvert_g2d ! queue ! waylandsink

 with the same results.