IMX8mp gstreamer vpuenc_h264 set profile and level

[TerryBarnaby1]

We are using an IMX8mp on a custom board and are using the NXP Yocto hardknott as the base platform.

We are compressing/encoding a camera stream to H264/MP4 using the hardware H264 encoders vpuenc_h264 plugin.

A basic example is like:

gst-launch-1.0 -v videotestsrc num-buffers=100 ! video/x-raw,width=720,height=576,framerate=25/1 ! vpuenc_h264 ! video/x-h264,profile=main ! h264parse ! mp4mux fragment-duration=10 ! filesink location=/tmp/Video1.mp4

However it appears that the vpuenc_h264 igores the profile= and level= stream parameters. Looking at the imx-gst1.0-plugin code there appears to be no ability to set the H264 profile.

1. Is there any ability to set the H264 hardware encoders profile using gstreamer parameters ?
2. I have seen some things on the Internet that appear to suggest you have to hardcode this setting into the VPU's hardware registers, is this the case ?
3. I note that an alternative (communit?) gstreamer1.0-plugins-imx plugin appears to have support for profile and level. I assume this cannot be used with the softwatre modules/kernel in the NXP fsl-nxp-bsp-hardknott Yocto build ?

[TerryBarnaby1 · Accepted solution]

Ok, thanks for the pointer.

Information for other users from what I have gathered searching the Internet and looking at the code as there is, typically, lack of NXP documentation available:

1. The H264 encoding profile is hard coded in vpu_wrapper to Baseline. You can change this to Mainline for example (rebuilding vpu_wrapper) and the H264 streams are marked as Mainline. If the actual compression is any different I don't know.
2. The level is set by vpu_wrapper based on the resolution of the video. Not sure if this has any affect other than tagging the H264 stream with this level. It looks like older versions of vpu_wrapper had a bug setting this level incorrectly.
3. If you don't specify a bitrate, at least at the gstreamer vpuenc_h264 level then the resulting stream will have a very low bit rate and thus be very low quality (at least with our video streams (generally static picture with occasional movements)). If not setting the bitrate ends up with the H264 encoder using some variable bitrate algorithm I don't know.

Feedback to NXP:

1. There needs to be documentation on the profile, level and bitrate of the H264 encoder, maybe in "i.MX Linux Reference Manual" or other places on this.
2. The NXP gstreamer vpuenc_h264 code should mention this in the code when listing information on the module with gst-inspect-1.0 as a lot of end users will be working at this level.

[Dhruvit]

Hi @TerryBarnaby1,

I hope you are doing well

Please find the information related to the same in the below thread.

https://community.nxp.com/t5/i-MX-Processors/Encoding-H-264-amp-H-265-using-vpu-wrapper-h/m-p/134761...

I hope this information helps you.

Thanks & Regards,

Dhruvit Vasavada