I am trying to test the VPU capability to encode a picture to JPEG. The i.Mx6 manual and datasheets state that it is possible to encode pictures up to 8192x8192 pixels in the VPU. I want to verify this but has not succeded due to memory allocation problems. Can anyone give me a suggestion how to verify and test the capability of i.Mx6 to JPEG encode a picture of size 8192x8192 in the VPU?
I have tested on a Nitrogen6_max board with 4GB of RAM. I am running Yocto on current master branch (1.8 WIP) with 3.10.53 kernel. On this plattform I have tested gstreamer0.10, gstreamer1.0 and mxc_vpu_test with the same result. In Gstream 1.0 the maximum resolution into
imxvpuenc_mjpeg is 1920x1080, any reason for this constraint?
Gstreamer 0.10:
GST_DEBUG=vpuenc:5 gst-launch-0.10 -v videotestsrc num-buffers=1 ! video/x-raw-yuv,format=\(fourcc\)UYVY,width=8192,height=8192,framerate=1/1,color-matrix=hdtv ! ffmpegcolorspace ! video/x-raw-yuv,format=\(fourcc\)NV12,width=8192,height=8192,framerate=1/1 ! vpuenc codec=mjpg ! filesink location=sample.jpg
From stdout:
0:00:02.042111334 678 0x101fd20 ERROR vpuenc /home/sepbe/yocto/nitrogen_master/build/tmp/work/nitrogen6x-poky-linux-gnueabi/gst-fsl-plugin/4.0.2-r0/gst-fsl-plugins-4.0.2/src/video/vpu/src/vpuenc.c:325:vpuenc_core_mem_alloc_dma_buffer: func VPU_EncGetMem failed!!
From /var/log/messages:
mxc_vpu 2040000.vpu: Physical memory allocation error!
mxc_vpu 2040000.vpu: Physical memory allocation error!
Mar 25 09:31:27 nitrogen6x user.err kernel: mxc_vpu 2040000.vpu: Physical memory allocation error!
Mar 25 09:31:27 nitrogen6x user.err kernel: mxc_vpu 2040000.vpu: Physical memory allocation error!
mxc_vpu_test:
root@nitrogen6x:~# VPU_TEST_DBG=5 ./mxc_vpu_test.out -E "-i /home/root/sample_8192_8192.yuv -o /home/root/images/image-205_yuv444.jpg -f 7 -c 1 -w 8192 -h 8192"
[INFO] main.c:515 VPU test program built on Mar 25 2015 09:24:14
[INFO] Product Info: i.MX6Q/D/S
[INFO] main.c:538 VPU firmware version: 3.1.1_r46061
[INFO] main.c:540 VPU library version: 5.4.27
[INFO] utils.c:535 Format: STD_MJPG
[INFO] utils.c:411 Input file "/home/root/sample_8192_8192.yuv" opened.
[INFO] utils.c:435 Output file "/home/root/images/image-205_yuv444.jpg" opened.
[INFO] enc.c:980 Capture/Encode fps will be 30
[DEBUG] enc.c:400 minfb 0
[ERR] mem allocation failed!
Frame buffer allocation failure
[ERR] enc.c:507 failed to allocate single framebuf
root@nitrogen6x:~#
Regards,
/Peter
Hi,
I am trying to get VPU accelerated MJPEG encode working. Input image is relatively large (4416 x 3312). I tried using libimxvpuapi but it turned out only support decoding MJPEG and no encoding available.
Can VPU handle this resolution?
Is mxc_vpu_test.out available as source?
Is there any alternative library/implementation?
Thanks,
Malek
Hi, Malek
I don't know what is your mean for 'no encoding available'
you can use the same vpu encode api as mpeg4/h.264 to encode mjpeg, just set different format.
mxc_vpu_test is open source, suppose you can get the source from our formal release.
The only work you need to do is adjust the memory pool to allow large memory allocation and set proper value for 'VPU_ENC_BITS_BUF_SIZE' accordingly just like mentioned above.
Eagle
Hi, thanks for you reply Peng
I am using this library: Freescale/libimxvpuapi · GitHub
And exactly as you said, I used the example given for h.264 encode and changed format to MJPG.
Program compiles OK, but when I ran it I got an error saying: invalid codec format
This is confirmed by looking at the switch statement (see link bellow), which only accept MPEG4, H263 and H264 formats for encoding.
Please look at the source, line 1407: libimxvpuapi/imxvpuapi_fslwrapper.c at master · Freescale/libimxvpuapi · GitHub
Is there any workaround?
Thanks,
Malek
Hi, Malek
For the libimxvpuapi, it was developed by third-party, you can ask help in corresponding community
or you can get freescal formal release package from freescal offical site.
Eagle
An update around my work to get JPEG encoding of high res pictures to work. Still no luck to get it working but I have come a bit further. A question that is still unanswered is if Freescale with any software ever verified the feature to encode 8192x8192 pictures to JPEG?
In order to avoid the memory allocation problem I have increased the CMA buffer inte the kernel. The problem I see right now is that I get a timeout from the VPU. I will describe and hope someone can give me some more input.
GST_DEBUG=vpuenc:5 VPU_LIB_DBG=5 gst-launch-0.10 -v videotestsrc num-buffers=1 ! video/x-raw-yuv,format=\(fourcc\)UYVY,width=6400,height=4800,framerate=1/1,color-matrix=hdtv ! ffmpegcolorspace ! video/x-raw-yuv,format=\(fourcc\)NV12,width=6400,height=4800,framerate=1/1 ! vpuenc codec=mjpg ! filesink location=sample.jpg
........
vpuenc versions
plugin: 4.0.3
wrapper: 1.0.58(VPUWRAPPER_ARM_LINUX Build on Apr 21 2015 15:40:06)
vpulib: 5.4.28
firmware: 3.1.1.46063
........
0:00:03.992470667 834 0x759f20 LOG vpuenc /home/sepbe/yocto/nitrogen_master/build/tmp/work/nitrogen6x-poky-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/video/vpu/src/vpuenc.c:1277:gst_vpuenc_chain: chain in with inbuffer size = 46080000 ts 0:00:00.000000000
vpu_lib.c:2068 enter vpu_EncGiveCommand()
vpu_lib.c:1523 enter vpu_EncStartOneFrame()
vpu_io.c:811 vpu clock gate setting = 1
vpu_io.c:811 vpu clock gate setting = 1
vpu_io.c:811 vpu clock gate setting = 0
vpu_lib.c:129 enter vpu_WaitForInt()
vpu_lib.c:132 ret of IOWaitForInt 0
vpu_io.c:811 vpu clock gate setting = 1
vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
vpu_lib.c:224 pic not done, wait
vpu_io.c:811 vpu clock gate setting = 0
vpu_lib.c:129 enter vpu_WaitForInt()
vpu_lib.c:132 ret of IOWaitForInt -1
vpu_io.c:811 vpu clock gate setting = 1
vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
vpu_lib.c:224 pic not done, wait
vpu_io.c:811 vpu clock gate setting = 0
vpu_lib.c:129 enter vpu_WaitForInt()
vpu_lib.c:132 ret of IOWaitForInt -1
vpu_io.c:811 vpu clock gate setting = 1
vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
vpu_lib.c:224 pic not done, wait
vpu_io.c:811 vpu clock gate setting = 0
vpu_lib.c:129 enter vpu_WaitForInt()
vpu_lib.c:132 ret of IOWaitForInt -1
vpu_io.c:811 vpu clock gate setting = 1
vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
vpu_lib.c:224 pic not done, wait
vpu_io.c:811 vpu clock gate setting = 0
vpu_lib.c:129 enter vpu_WaitForInt()
vpu_lib.c:132 ret of IOWaitForInt -1
vpu_io.c:811 vpu clock gate setting = 1
vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000
vpu_lib.c:224 pic not done, wait
vpu_io.c:811 vpu clock gate setting = 0
0:00:24.082916670 834 0x759f20 ERROR vpuenc /home/sepbe/yocto/nitrogen_master/build/tmp/work/nitrogen6x-poky-linux-gnueabi/gst-fsl-plugin/4.0.3-r0/gst-fsl-plugins-4.0.3/src/video/vpu/src/vpuenc.c:1328:gst_vpuenc_chain: func VPU_EncEncodeFrame failed!! with ret 8
It seems that it is a timeout from the VPU. Inside the function IOWaitForInt the status is traced out. The value of status is read from register MJPEG_PIC_STATUS_REG.
[DEBUG] | vpu_lib.c:219 status 0x4, wrPtr 0x46300000, bbcEnd 0x46300000 |
#define MJPEG_PIC_STATUS_REG | (NPT_BASE + 0x004) | // [3] - overflow, [2] - bbc interrupt, [1] - error, [0] - done |
How shall I interpret status=0x4? I can not find any register definition document of the MJPEG_PIC_STATUS_REG.
I am quite stuck here. Where can I find more information regarding this problem? With my setup I have been able to encode a picture of size 5120x3200, increasing the reolution from there I always end up in this timeout situation.
Best regards,
/Peter
Dear Peter,
Thank you so much for your contributions on this matter! You really dug deep into the issue and this is greatly appreciated.
I have a question for you: do you by any chance remember how you set the CMA buffer size in the kernel? If I pass cma=<value>M to the bootargs in the 6x_bootscript, it does not boot. Also, what size did you set the buffer to?
Kindest regards,
Cristian Pandele
Nevermind, I figured it out myself. In the .config file for the kernel, this entry controls the maximum allocable contiguous memory size: CONFIG_CMA_SIZE_MBYTES. The initial value was 256, I changed it to 512 and it seems to work fine.
Thank you once again for navigating these deep waters and providing a useful workaround!
It seems that there is a limitation of the output buffer to 1MB in the libfslvpuwrap.
Regards,
/Peter
Hi, Peter
Yes, buffer default size 'VPU_ENC_BITS_BUF_SIZE' maybe one risk for your case.
you had better enlarge it accordingly
Thanks.
Eagle
Hi Peng,
I too faced same issue(VPU Timeout) while performing VPU Encoding at resolution(3264x2448). On Increasing the buffer size(VPU_ENC_BITS_BUF_SIZE) to 4MB, the issue got solved. Does this affects the performance of VPU in any manner?.
How this buffer size related with the timeout issue?
Regards,
Teddy
Hi, Teddy
The macro 'VPU_ENC_BITS_BUF_SIZE' is just the size for output data buffer, and won't impact performance.
If the buffer size is not big enough, it may lead to potential risk, such as memory overflow or vpu timeout
Eagle
Hi, Peter,
Sorry for late reply !
Due to the limitation of allocating physically contiguous memory space, the 8Kx8K JPEG encode/decode has not been supported yet in our BSP. For example for YUV444 format, it needs 64x3=192Mbytes contiguous memory which is difficult to achieve. For "vpuenc" gst glugin, we currently support only up to 1080p resolution for JPEG. We are planning to increase the resolution but the maximum resolution has not been determined yet.
May I ask what is the use case for using the JPEG encode/decode at 8Kx8K resolution ?
Thanks
Jones
Hi,
thanks for your reply and the information given. It confirms the results I have got.
The use case is to use the JPEG encoder inside i.Mx6 in a high quality still image camera with resolutions in the range 40-60 mexapixel resolution. The plan was to encode pictures of size 8192x6144 to JPEG, according to i.Mx6's datasheet this should be possible.
What is your feeling of the effort needed to develop a driver to Linux that can do JPEG encoding of 8192x6144 in i.Mx6 running Linux?
Best regards,
/Peter