hantro h264 broken in kernel 6.6.52

[ihermida]

Hi all,

There is a regresion on a kernel v6.6 due to commit https://github.com/nxp-imx/linux-imx/commit/4ca7e59f00e03acb01476e106de3870214efc682

You can try the next command to test the h264 codec, or well reproduce a video containing it.

gst-launch-1.0 videotestsrc ! video/x-raw, format=I420, width=640, height=480 ! vpuenc_h264 ! h264parse ! v4l2h264dec ! fakesink

Reverting that commit solves the problem, but the real problem is in the file drivers/mxc/hantro_v4l2/vsi-v4l2.h, where adding a new field in the "struct v4l2_daemon_dec_buffers " causes a missaligment (in the specific commit it is the field "u64 process_time")

Can you point to a proper fix for having h264 working?

BR / Isaac

[joanxie]

do you use imx8mm? what kind of issue do you find when you use this command? I haven't got imx8mm now, pls share the detailed issue or logfile with me

[ihermida]

Yes, I reproduce it in the i.mx8m mini. We have seen it using default NXP images for Android, and I narrow it down till linux and the mentioned commit.

You can monitor the "hantrodec" interrupts and see how they are not generated. and the video is not reproduced. Try the command 

gst-launch-1.0 videotestsrc ! video/x-raw, format=I420, width=640, height=480 ! vpuenc_h264 ! h264parse  ! v4l2h264dec ! fakesink

(or fpsdisplaysink).

The error is that the command just get locked (no video, no interrupts, nothing in dmesg).

"""

root@ccimx8mm-dvk:~# gst-launch-1.0 videotestsrc ! video/x-raw, format=I420, width=640
, height=480 ! vpuenc_h264 ! h264parse  ! v4l2h264dec ! fakesink
Setting pipeline to PAUSED ...

====== V4L2DEC: 1.24.0 build on Jul 24 2024 02:25:45. ======
====== VPUENC: 4.9.1 build on Aug  8 2024 05:28:21. ======
       wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jul 24 2024 01:56:50)
       vpulib: 1.1.1
       firmware: 1.1.1.65535
Pipeline is PREROLLING ...
Redistribute latency..
""

You can use gplay and a video with h264 codec. Gplay just fails as it cannot reproduce the video. Use whatever command to reproduce the problem. And check the interrupts.

root@ccimx8mm-dvk:~# grep hantro /proc/interrupts
29:          0          0          0          0     GICv3  39 Level     hantrodec
30:          0          0          0          0     GICv3  40 Level     hantrodec
root@ccimx8mm-dvk:~#

In case of internal vpu error, it fails "silently" so there is not any relevant information in the dmesg output.

The error using a h264 video with gplay is "a general stream error"

"""

ERROR Error from element /GstPlayBin3:playbin3/GstURIDecodeBin3:uridecodebin3/GstURISo
urceBin:urisourcebin0/GstParseBin:parsebin0/GstAiurDemux:aiurdemux0: GStreamer encount
ered a general stream error.
GStreamer encountered a general stream error.
/usr/src/debug/imx-gst1.0-plugin/4.9.1+git/plugins/aiurdemux/aiurdemux.c(1229): aiurde
mux_pull_task (): /GstPlayBin3:playbin3/GstURIDecodeBin3:uridecodebin3/GstURISourceBin
:urisourcebin0/GstParseBin:parsebin0/GstAiurDemux:aiurdemux0:
streaming stopped, reason not-negotiated, state 3 for /mnt/data/3DMark_1280x720_h264_a
ac_44100
"""

[ihermida · Accepted solution]

I found where the problem comes...

We were using the meta-freescale layer, which is using the imx-vpu-hantro version 1.32.

This recent commit in the layer solved the problem

https://github.com/Freescale/meta-freescale/commit/a63a107a3681d4a6342145290f62e98d8607ec0f

It might be a missaligment between the kernel (that introduced the u64 process_time value) and the imx vpu code.

Using the kernel 6.6.52 with the imx vpu hantro 1.35 works.