iMX8QXP: Use JPEG decoder in M4 SDK

[#1]

iMX8QXP/iMX8QM have hardware JPEG decoder: The JPEG-D-X core. This is the example code to use this hw decoder in M4 SDK to decode JPEG files.

M4_JPEG_DECODER_SDK_2.5.1.7z

The attached "rear_view_camera_jpegdec.tar.bz2" is the updated source code for "SDK\boards\mekmimx8qx\demo_apps\rear_view_camera". It is based on SDK 2.5.1 for iMX8QXP MEK.

The "rear_view_camera_jpegdec.patch" is the modified code, it hasn't included the added "fsl_jpeg_dec.c" and "fsl_jpeg_dec.h".

The testing used two 256*256 JPEG files, they are RGB color space. We used followed commands to build them into flash.bin:

./mkimage_imx8 -soc QX -rev B0 -append ahab-container.img -c -scfw scfw_tcm.bin -m4 m4_rear_view_camera.bin 0 0x34FE0000 --data demo_rgb.jpg 0x84000000 --data demo_rgb2.jpg 0x84008000 -out flash.bin

If customer need change the JPEG resoluion, they can change them in file "fsl_jpeg_dec.h", APP_JPEG_SIZE_OF_KB is the JPEG file length in memory, aligned in KB.
 

#define APP_JPEG_WIDTH (256)
#define APP_JPEG_HEIGHT (256)
#define APP_JPEG_SIZE_OF_KB (32)
#define APP_JPEG_FORMAT JPEG_RGB

#define APP_JPEG_BUFFER (0x84000000)

To created RGB format JPEG file from RGB data, the customer can use linux unit test application "/unit_tests/JPEG/encoder_test.out".

M4_JPEG_DECODER_WINDOW_MODE_SDK_2.5.1.7z

Based on JEPG decoder, added DPU CSC support and render JEPG decoded video in overlay window.

The architecture is followed: NXP logo is put in FetchLayer0 with RGB565 format, after LayerBlend0, it will be the prim layer for LayberBlend1 (FetchLayer0 can't be used as prim layer for LayerBlend), the JPEG decoder output is put to FetchDecoder0. RGB888 format, and it will be resize to 640*480, and put to x=100, y=100 of the display. (Only the sec layer of LayerBlend can be window mode).

Some limitation for layer selection in LayerBlend:

[hannahunderwood]

I am using the Media SDK 2014 R2 for Clients that I recently downloaded from the Intel site.  I also downloaded the Intel Media SDK Samples and built the Video Decoding Sample and was able to successfully decode and render an output image as well as convert the resulting output to YUV frames My Lowes Life.

[qiang_li-mpu_se]

This reference decoder code is based on the hardware JPEG decoder in iMX8QXP chip, it is not software based decoder.

[songdz]

where can get the tools("/unit_tests/JPEG/encoder_test.out").

[qiang_li-mpu_se]

If you had built the Yocto BSP successfully, in the download folder, there is a package "linux-test-xxx.tar.gz", it includes all unit tests codes.

[songdz]

I am porting according to your patch. Found that M4 is directly attached to APP_InitJpeg_Dec(), then I applied for SC_R_MJPEG_DEC_MP and SC_R_MJPEG_DEC_S0 in scfw. M4 can run, but there is no picture display, and the JPEG interrupt program does not run.

[qiang_li-mpu_se]

The memory buffer used in M4 for jpeg file and decoder should also be checked. They should be assigned to M4 in SCFW too.

[songdz]

Can you send me the relevant configuration of SCFW?

[qiang_li-mpu_se]

For shared memory related issue, you can reference to Android BSP and https://community.nxp.com/docs/DOC-343706.

[songdz]

[songdz]

[songdz]

I found that after decoding the jpg image, the color of the displayed image is incorrect

[qiang_li-mpu_se]

There are many different kinds of JPEG files, you use set the correct one in M4:

 JPEG_YUV420 = 0x0, /* 2 Plannar, Y=1st plane UV=2nd plane */
 JPEG_YUV422 = 0x1, /* 1 Plannar, YUYV sequence */
 JPEG_RGB = 0x2, /* RGBRGB packed format */
 JPEG_YUV444 = 0x3, /* 1 Plannar, YUVYUV sequence */
 JPEG_GRAY = 0x4, /* Y8 or Y12 or Single Component */
 JPEG_RESERVED = 0x5,
 JPEG_ARGB = 0x6,

[jinchen1227]

Are there any patch for imx8qm-mek?

[qiang_li-mpu_se]

Hi Chen King

The JPEG module is same in iMX8QM and iMX8QXP, so the code can be used on iMX8QM too.

[xiaoqianlin]

Looks good, thank you. 

[Aubineau_FAE]

Thank you @qiang_li-mpu_se for sharing, works like a charm.

For those using windows/WSL, you can convert the 2 patched file in Linux text format, in Linux fil format, in your WSL Linux:

$ sudo apt-get install dos2unix
$ dos2unix devices/MIMX8QX6/drivers/fsl_dpu.c
$ dos2unix devices/MIMX8QX6/drivers/fsl_dpu.h
$ patch -p1 < mcu_sdk_dpu_driver.patch

If you use a Lauterbach probe you can also avoid using mkimage, but just load jpeg images thru JTAG by adding the following command in your cmm file:

data.load.binary "256x256_1.jpg" 0x84000000
data.load.binary "256x256_2.jpg" 0x84008000

I have monitored the decode time for a 1280x480 file and it is quite fast, including OS overhead and display, it last #5.1ms: