i.MX8QM VPU information needed for developing H.264 decoding software

[ToarteFretter]

Dear,

We would like to develop software to decode H.264 video using the VPU inside the i.MX8QM.

However, the reference manual is very brief on the VPU details.

Is there more information available regarding its operation; for example, how to interact with these four M0+ cores, how to provide the TS data, how to retrieve the decoded video?

Thanks in advance!

Best regards.

[Manuel_Salas]

Hello @ToarteFretter 

Hope you are doing very well.

Unfortunately, NXP does not publicly share detailed Amphion VPU firmware documentation. The firmware is provided by the IP owner (Amphion), and NXP does not have any driver or SDK examples for controlling the VPU directly from the M core.

We just have the i.MX VPU Application Programming Interface Linux Reference Manual available.

Best regarsd,

Salas.

[ToarteFretter]

Hi @Manuel_Salas,

Can you tell us if the firmware that needs to run on the M0+ cores for the VPU functionality on the i.MX8QM, has to be loaded by the software running on the A-core (e.g.: by Linux), or is this baked into the chip through ROM or maybe part of the SCFW?

Quickly looking into the source code of Linux doesn't give us a clear response on this question; also because of the confusing and many flavours of the i.MX8, as well as the VPU types from what it looks.

It would be very helpful of you, if you could give some more information on this? Also, if firmware needs to be loaded by us, where that we can find it?

Many thanks!

Best regards,

ToarteFretter.

[Manuel_Salas · Accepted solution]

Hello @ToarteFretter 

For the i.MX8QM Amphion VPU, the decoder firmware is not stored in ROM and is not part of SCFW.

It is a separate binary that is loaded at runtime by the A-core host software (by the Linux Amphion kernel driver).

Also, you can find the binary with:

wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/firmware-imx-8.33-7ce4783.bin

Inside, you will find the vpu_fw_imx8_dec.bin and vpu_fw_imx8_enc.bin.

NOTE: Table from i.MX Linux Release Notes.

Best regards,

Salas.

[ToarteFretter]

Hello @Manuel_Salas,

Can it be that next to the decoder and encoder firmware's, that run respectively on one and two M0+ cores, there is also a third firmware needed for the fourth M0+ core?

If yes, could this be ‘vpu_fw_imx8_xuvi.bin’ inside the provided archive?

Many thanks!

Best regards,

Toartefretter.

[Manuel_Salas]

Hello.

Correct.

VPU core	Function	Firmware
M0+ core 0	Malone decoder	vpu_fw_imx8_dec.bin
M0+ cores 1–2	Windsor encoder	vpu_fw_imx8_enc.bin
M0+ core 3	XUVI/PPM processing, including TS-related functions	vpu_fw_imx8_xuvi.bin

Best regards,

Salas.

[ToarteFretter]

Hello @Manuel_Salas,

Many thanks!

Regards,

Toartefretter.

[ToarteFretter]

Hello @Manuel_Salas

Yes, we're doing fine, thanks for asking!

Just to be sure we're aligned, we want to develop the software on the A-core, not on the M4 or even directly on the M0+ cores.

From what we've seen in the provided document, there should be a driver inside the Linux distribution, right? I think I've found the source code of this.

The firmware itself that must be loaded on these M0+ cores, is this also somewhere present inside the Linux distribution?

Many thanks again!

Best regards,

ToarteFretter.