JPEG hardware decode and resize in i.MX6

[shaojun_wang]

MX6 VPU can support baseline JPEG decode and encode. This demo show how to use VPU to decode baseline JPEG and how to use G2D to convert, crop and resize the VPU output image.

The BSP version in this demo is L3.0.35_4.1.0.

1. VPU decode

1.1 VPU lib patch

VPU can support JPEG decode and downscale, the downscale rate is 0, 1/2, 1/4 and 1/8.

0001-support-jpeg-downscale-in-vpu-lib.patch: VPU lib patch to support JPEG decode and downscale.

Run below step to extract VPU lib source code from ltib, apply patch and build new VPU lib

$ ./ltib -m prep -p imx-vpu

$ Apply 0001-support-jpeg-downscale-in-vpu-lib.patch to ltib/rpm/BUILD/imx-vpu-lib-3.0.35-4.1.0

$ ./ltib -m scbuild -p imx-vpu

$ Update ltib/rpm/BUILD/imx-vpu-lib-3.0.35-4.1.0/libvpu.so.4 to your rootfs.

1.2 Kernel patch

When run stress test, VPU may not get DMA buffer. The decode would failed.

0001-VPU-Reserve-64MB-physical-memory-for-VPU.patch can fix the memory allocate failed issue, it reserve 64MB physical memory for VPU.

1.3 JPEG file size and VPU output image size

In L3.0.35_4.1.0, CONFIG_FORCE_MAX_ZONEORDER is set to 14, so the max DMA buffer size is 2^(14-1)*4094=32MB.

If your Kernel CONFIG_FORCE_MAX_ZONEORDER is less than 14, please modify STREAM_BUF_SIZE in mxc_jpeg_test/jpeg_test.h to the correct value.

In VPU lib, two DMA buffers are allocated, one is input buffer, for JPEG source file; the other is output buffer, for VPU output raw image. So the JPEG file size should not larger than 32MB. This demo use VPU downscale function to limit output image size to less than 32MB.

2. G2D

Since VPU output format is YUV, to show the image to display, need to convert the format to RGB and resize.

This demo use G2D to do the conversion. Make sure gpu-viv-bin-mx6q is built into your rootfs.

About more information of G2D, please refer to i.MX_6_G2D API.pdf.

In ltib, extract GPU lib by

$ ./ltib -m prep -p gpu-viv-bin-mx6q

3. Build the demo

In mxc_jpeg_test/Makefile, modify INC and LFLAGS to point to the correct path of Linux kernel, VPU lib and GPU lib, then run below command

$ export CROSS_COMPILE=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-

$ make clean; make

4. Run the demo

4.1 Get help of this demo

$ ./mxc_jpeg_test.out -H

4.2 Decode and display in a window

$ ./mxc_jpeg_test.out -D "-i 1.jpg -w 640 -h 480 -l 200 -t 200"

4.4 Enable zoom and drag effect on small window, rotate 90

$ ./mxc_jpeg_test.out -D "-i 1.jpg -z 29 -w 320 -h 240 -l 200 -t 200 -r 1"

5. VPU limitation

5.1 There are two kinds of JPEG format, Baseline and Progressive. VPU only support Baseline format.

5.2 VPU can’t decode JPEG file with width or height larger than 8192

Original Attachment has been moved to: 0001-support-jpeg-downscale-in-vpu-lib.patch.txt.zip

Original Attachment has been moved to: libvpu.so.4.zip

Original Attachment has been moved to: mxc_jpeg_test.tgz

Original Attachment has been moved to: 0001-VPU-Reserve-64MB-physical-memory-for-VPU.patch.txt.zip

[yongtang]

What is the mean of "Up-scaling and down-scaling from source to destination",is downscale the file size or image resolution?

[宗標廖]

Hello Shaojun

You said that :

>5.1 There are two kinds of JPEG format, Baseline and Progressive. VPU only support Baseline format.

So if I want to decode a JPEG file with Progressive format, how should I do.

Thank you!

ZongbiaoLiao

[alik]

Hello,

How can the VPU-Reserve-64MB-physical-memory patch be modified for newer kernels that use device tree and do not use board files? Where the dma_declare_coherent_memory() call can be placed?

Thanks,

Alex

[alik]

To answer my own question, the VPU-Reserve patch could be modified, the file to edit is mach-imx6q.c and it has board init function and reserve function can be added to port the patch to the device tree kernels. However, that patch did not help with the "Physical memory allocation error!" that I intermittently had when using VPU to decode large JPEG images that required 70MB buffers. Upon further investigation, it turned out that CMA is actually being used to allocate DMA memory in vpu_lib L3.10.17 kernel that I am using. The default kernel config in my case had CMA size set at 256M, but that was not enough. Increasing CMA to a larger value (I tried 512M so far) through cmm=512M kernel command line parameter helped.

Alex

[Danube]

Hi  Shaojun,

Do you have any example for 3.10.53 BSP ?

[tongchunyang]

Hi,Shaojun

How can I decode the picture with MODE224?

I try to modify your code like this:

---g2d.c---

case MODE422:

case MODE224://add

        src.format = NV16;

        pfb_dec=dec->pfbpool[0];

        break;

---

Unfortunately,the color is strange.

I'm looking forward to your replay.

[tongchunyang]

Hi,Shaojun

How can I decode the picture with MODE224?

I try to modify your code like this:

---g2d.c---

case MODE422:

case MODE224://add

        src.format = NV16;

        pfb_dec=dec->pfbpool[0];

        break;

---

Unfortunately,the color is strange.

I'm looking forward to your replay.

[shaojun_wang]

The mxc_jpeg_test.tgz has been upgraded to support to MODE224.

[tongchunyang]

Thank you very much!

[takezoumurakami]

Hi Mr Shaojun,

I 've tried the code "mxc_jpeg_test.tgz",but I'm getting errors,when I executed.

The errors is following.

"g2d_open: fail with status -24"

I think "status -24" is version miss match between g2d library version and kernel version.

How can I solve error?

Best regards.

[shaojun_wang]

First you need to build gpu-viv-bin-mx6q into your rootfs.

Then modiry the Makefile to link to your g2d lib in rootfs.

[远李]

Hi Shaojun,

How can I download i.MX_6_G2D API.pdf?

[shaojun_wang]

You can find it in Ltib release doc, such as L3.0.35_4.1.0_docs.tar.gz.