VPU MJPEG encode 8192x8192

[PeterBergin]

I am trying to test the VPU capability to encode a picture to JPEG. The i.Mx6 manual and datasheets state that it is possible to encode pictures up to 8192x8192 pixels in the VPU. I want to verify this but has not succeded due to memory allocation problems. Can anyone give me a suggestion how to verify and test the capability of i.Mx6 to JPEG encode a picture of size 8192x8192 in the VPU?

I have tested on a Nitrogen6_max board with 4GB of RAM. I am running Yocto on current master branch (1.8 WIP) with 3.10.53 kernel. On this plattform I have tested gstreamer0.10, gstreamer1.0 and mxc_vpu_test with the same result. In Gstream 1.0 the maximum resolution into

imxvpuenc_mjpeg is 1920x1080, any reason for this constraint?

Gstreamer 0.10:

GST_DEBUG=vpuenc:5 gst-launch-0.10 -v videotestsrc num-buffers=1 ! video/x-raw-yuv,format=\(fourcc\)UYVY,width=8192,height=8192,framerate=1/1,color-matrix=hdtv ! ffmpegcolorspace ! video/x-raw-yuv,format=\(fourcc\)NV12,width=8192,height=8192,framerate=1/1 ! vpuenc codec=mjpg ! filesink location=sample.jpg

From stdout:

0:00:02.042111334   678  0x101fd20 ERROR                 vpuenc /home/sepbe/yocto/nitrogen_master/build/tmp/work/nitrogen6x-poky-linux-gnueabi/gst-fsl-plugin/4.0.2-r0/gst-fsl-plugins-4.0.2/src/video/vpu/src/vpuenc.c:325:vpuenc_core_mem_alloc_dma_buffer: func VPU_EncGetMem failed!!

From /var/log/messages:

mxc_vpu 2040000.vpu: Physical memory allocation error!
mxc_vpu 2040000.vpu: Physical memory allocation error!
Mar 25 09:31:27 nitrogen6x user.err kernel: mxc_vpu 2040000.vpu: Physical memory allocation error!
Mar 25 09:31:27 nitrogen6x user.err kernel: mxc_vpu 2040000.vpu: Physical memory allocation error!

mxc_vpu_test:

root@nitrogen6x:~# VPU_TEST_DBG=5 ./mxc_vpu_test.out -E "-i /home/root/sample_8192_8192.yuv -o /home/root/images/image-205_yuv444.jpg -f 7 -c 1 -w 8192 -h 8192"
[INFO] main.c:515 VPU test program built on Mar 25 2015 09:24:14
[INFO] Product Info: i.MX6Q/D/S
[INFO] main.c:538 VPU firmware version: 3.1.1_r46061
[INFO] main.c:540 VPU library version: 5.4.27
[INFO] utils.c:535 Format: STD_MJPG
[INFO] utils.c:411 Input file "/home/root/sample_8192_8192.yuv" opened.
[INFO] utils.c:435 Output file "/home/root/images/image-205_yuv444.jpg" opened.
[INFO] enc.c:980 Capture/Encode fps will be 30
[DEBUG] enc.c:400 minfb 0
[ERR] mem allocation failed!
Frame buffer allocation failure
[ERR] enc.c:507 failed to allocate single framebuf
root@nitrogen6x:~# 

Regards,

/Peter

[maleklamari]

Hi,

I am trying to get VPU accelerated MJPEG encode working. Input image is relatively large (4416 x 3312). I tried using libimxvpuapi but it turned out only support decoding MJPEG and no encoding available.

Can VPU handle this resolution?

Is mxc_vpu_test.out available as source?

Is there any alternative library/implementation?

Thanks,

Malek

[eaglezhou]

Hi, Malek

   I don't know what is your mean for 'no encoding available'

   you can use the same vpu encode api as mpeg4/h.264 to encode mjpeg, just set different format.

   mxc_vpu_test is open source, suppose you can get the source from our formal release.

   The only work you need to do is adjust the memory pool to allow large memory allocation and set proper value for 'VPU_ENC_BITS_BUF_SIZE' accordingly just like mentioned above.

Eagle

[maleklamari]

Hi, thanks for you reply Peng

I am using this library: Freescale/libimxvpuapi · GitHub

And exactly as you said, I used the example given for h.264 encode and changed format to MJPG.

Program compiles OK, but when I ran it I got an error saying: invalid codec format

This is confirmed by looking at the switch statement (see link bellow), which only accept MPEG4, H263 and H264 formats for encoding.

Please look at the source, line 1407: libimxvpuapi/imxvpuapi_fslwrapper.c at master · Freescale/libimxvpuapi · GitHub

Is there any workaround?

Thanks,

Malek

[eaglezhou]

Hi, Malek

   For the libimxvpuapi, it was developed by third-party, you can ask help in corresponding community

   or you can get freescal formal release package from freescal offical site.

Eagle

[PeterBergin]

An update around my work to get JPEG encoding of high res pictures to work. Still no luck to get it working but I have come a bit further. A question that is still unanswered is if Freescale with any software ever verified the feature to encode 8192x8192 pictures to JPEG?

In order to avoid the memory allocation problem I have increased the CMA buffer inte the kernel. The problem I see right now is that I get a timeout from the VPU. I will describe and hope someone can give me some more input.

GST_DEBUG=vpuenc:5 VPU_LIB_DBG=5 gst-launch-0.10 -v videotestsrc num-buffers=1 ! video/x-raw-yuv,format=\(fourcc\)UYVY,width=6400,height=4800,framerate=1/1,color-matrix=hdtv ! ffmpegcolorspace ! video/x-raw-yuv,format=\(fourcc\)NV12,width=6400,height=4800,framerate=1/1 ! vpuenc codec=mjpg ! filesink location=sample.jpg

........

vpuenc versions
  plugin: 4.0.3
  wrapper: 1.0.58(VPUWRAPPER_ARM_LINUX Build on Apr 21 2015 15:40:06)
  vpulib: 5.4.28
  firmware: 3.1.1.46063

........

0:00:03.992470667   834   0x759f20 LOG                   vpuenc /home/sepbe/yocto/nitrogen_master/build/tmp/work/nitrogen6x-poky-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/video/vpu/src/vpuenc.c:1277:gst_vpuenc_chain: chain in with inbuffer size = 46080000 ts 0:00:00.000000000
  vpu_lib.c:2068 enter vpu_EncGiveCommand()
  vpu_lib.c:1523 enter vpu_EncStartOneFrame()
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_io.c:811 vpu clock gate setting = 0
  vpu_lib.c:129 enter vpu_WaitForInt()
  vpu_lib.c:132 ret of IOWaitForInt 0
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
  vpu_lib.c:224 pic not done, wait
  vpu_io.c:811 vpu clock gate setting = 0
  vpu_lib.c:129 enter vpu_WaitForInt()
  vpu_lib.c:132 ret of IOWaitForInt -1
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
  vpu_lib.c:224 pic not done, wait
  vpu_io.c:811 vpu clock gate setting = 0
  vpu_lib.c:129 enter vpu_WaitForInt()
  vpu_lib.c:132 ret of IOWaitForInt -1
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
  vpu_lib.c:224 pic not done, wait
  vpu_io.c:811 vpu clock gate setting = 0
  vpu_lib.c:129 enter vpu_WaitForInt()
  vpu_lib.c:132 ret of IOWaitForInt -1
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
  vpu_lib.c:224 pic not done, wait
  vpu_io.c:811 vpu clock gate setting = 0
  vpu_lib.c:129 enter vpu_WaitForInt()
  vpu_lib.c:132 ret of IOWaitForInt -1
  vpu_io.c:811 vpu clock gate setting = 1
  vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
  vpu_lib.c:224 pic not done, wait
  vpu_io.c:811 vpu clock gate setting = 0
0:00:24.082916670   834   0x759f20 ERROR                 vpuenc /home/sepbe/yocto/nitrogen_master/build/tmp/work/nitrogen6x-poky-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/video/vpu/src/vpuenc.c:1328:gst_vpuenc_chain: func VPU_EncEncodeFrame failed!! with ret 8

It seems that it is a timeout from the VPU. Inside the function IOWaitForInt the status is traced out. The value of status is read from register MJPEG_PIC_STATUS_REG.

[DEBUG]

vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000

#define MJPEG_PIC_STATUS_REG

(NPT_BASE + 0x004)

// [3] - overflow, [2] - bbc interrupt, [1] - error, [0] - done

How shall I interpret status=0x4? I can not find any register definition document of the MJPEG_PIC_STATUS_REG.

I am quite stuck here. Where can I find more information regarding this problem? With my setup I have been able to encode a picture of size 5120x3200, increasing the reolution from there I always end up in this timeout situation.

Best regards,

/Peter

[cristianpandele]

Dear Peter,

Thank you so much for your contributions on this matter! You really dug deep into the issue and this is greatly appreciated.

I have a question for you: do you by any chance remember how you set the CMA buffer size in the kernel? If I pass cma=<value>M to the bootargs in the 6x_bootscript, it does not boot. Also, what size did you set the buffer to?

Kindest regards,

Cristian Pandele

[cristianpandele]

Nevermind, I figured it out myself. In the .config file for the kernel, this entry controls the maximum allocable contiguous memory size: CONFIG_CMA_SIZE_MBYTES. The initial value was 256, I changed it to 512 and it seems to work fine.

Thank you once again for navigating these deep waters and providing a useful workaround!

[PeterBergin]

It seems that there is a limitation of the output buffer to 1MB in the libfslvpuwrap.

Regards,

/Peter

[eaglezhou]

Hi, Peter

     Yes, buffer default size 'VPU_ENC_BITS_BUF_SIZE' maybe one risk for your case.

     you had better enlarge it accordingly

     Thanks.

Eagle

[teddy123]

Hi Peng,

I too faced same issue(VPU Timeout) while performing VPU Encoding at resolution(3264x2448). On Increasing the buffer size(VPU_ENC_BITS_BUF_SIZE) to 4MB, the issue got solved. Does this affects the performance of VPU in any manner?.

How this buffer size related with the timeout issue?

Regards,

Teddy

[eaglezhou]

Hi, Teddy

    The macro 'VPU_ENC_BITS_BUF_SIZE' is just the size for output data buffer, and won't impact performance.

    If the buffer size is not big enough, it may lead to potential risk, such as memory overflow or vpu timeout

Eagle

[ZhongliHe]

Hi, Peter,

Sorry for late reply !

Due to the limitation of allocating physically contiguous memory space, the 8Kx8K JPEG encode/decode has not been supported yet in our BSP.  For example for YUV444 format, it needs 64x3=192Mbytes contiguous memory which is difficult to achieve. For "vpuenc" gst glugin, we currently support only up to 1080p resolution for JPEG. We are planning to increase the resolution but the maximum resolution has not been determined yet.

May I ask what is the use case for using the JPEG encode/decode at 8Kx8K resolution ?

Thanks

Jones

[PeterBergin]

Hi,

thanks for your reply and the information given. It confirms the results I have got.

The use case is to use the JPEG encoder inside i.Mx6 in a high quality still image camera with resolutions in the range 40-60 mexapixel resolution. The plan was to encode pictures of size 8192x6144 to JPEG, according to i.Mx6's datasheet this should be possible.

What is your feeling of the effort needed to develop a driver to Linux that can do JPEG encoding of 8192x6144 in i.Mx6 running Linux?

Best regards,

/Peter