I.MX6 - H.264 encoding of Framebuffer

[anoopaugustine]

Hi, I am trying to H.264 encode the framebuffer in i.MX6Q. My requirement is to encode and stream the framebuffer. I am able to open /dev/fb0 and grab the frame. I understand from the H.26 encoder requirements that t operates only on YUV frames. I am trying color space conversion of RGB framebuffer using IPU and then encoding using the VPU. But all these operations takes more than 100ms which I am not able to afford. Could you please suggest me a better way to do this? Is there a way I can configure /dev/fb0 for YUV and H.264 encoder can directly operate on framebuffer memory? Is there any samples available?

[anoopaugustine]

Hi Igor,

Thanks for the response.

I am using the following encode and decode example code.
https://github.com/Freescale/libimxvpuapi/blob/master/example/encode-example.c
https://github.com/Freescale/libimxvpuapi/blob/master/example/decode-example.c

I am trying to H264 encode a single frame (1920 * 1080 in I420 format). Encoding is succeeded using the encode example (I have updated frame width and height as 1920* 1080 in the example code). But while decoding using decode example I get following error and it is not producing decoded image file. What could be the problem here? Is it not possible to decode a single frame.

  [INFO]  bitstreamMode 1, chromaInterleave 0, mapType 0, tiled2LinearEnable 0
encoded input frame:  frame id: 0x64  size: 320710 byte
initial info:  size: 1920x1088 pixel  rate: 4294967295/4294967295  min num required framebuffers: 8  interlacing: 0  framebuffer alignment: 1
calculated sizes:  frame width&height: 1920x1088  Y stride: 1920  CbCr stride: 960  Y size: 2088960  CbCr size: 522240  MvCol size: 522240  total size: 3655681
draining decoder
VPU reports EOS; no more decoded frames available

Thanks & Regards

Anoop

[igorpadykov]

Hi Anoop

libimxvpuapi is community api, please post its issues on

meta-fsl-arm mailing list, so that someone familiar with it could try to assist you.
https://lists.yoctoproject.org/listinfo/meta-freescale

also one can look at vpu examples in unit tests

imx-test
www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-test-5.7.tar.gz

Best regards
igor

[anoopaugustine]

Hi Igor,

Thanks for your support.

I tried out the VPU H.264 unit test. I am seeing  ~30fps for encoding 1920*1080 frame. But for decoding it is showing fps as ~6fps. It takes about 162ms for decoding a single frame. That is way too slow. What could be the issue?

root@imx6qsabreauto:/unit_tests# ./mxc_vpu_test.out -E "-i /home/root/converted.yuv -f 2 -a 25 -o te...
[INFO]    VPU test program built on Jun 19 2017 11:13:06
[INFO]    Product Info: i.MX6Q/D/S
[INFO]    VPU firmware version: 3.1.1_r46072
[INFO]    VPU library version: 5.4.35
[INFO]    Format: STD_AVC
[INFO]    AVC
[INFO]    Input file "/home/root/converted.yuv" opened.
[INFO]    Output file "test1.h264" opened.
[INFO]    Capture/Encode fps will be 25
[INFO]    ringBufferEnable 0, chromaInterleave 1, mapType 0, linear2TiledEnable 0
[INFO]    Finished encoding: 1 frames
[INFO]    enc fps = 44.92
[INFO]    total fps= 30.77

root@imx6qsabreauto:/unit_tests# ./mxc_vpu_test.out -D "-i test1.h264 -f 2 -a 25 -o test1.yuv -w 1920 -h 1080"
[INFO]    VPU test program built on Jun 19 2017 11:13:06
[INFO]    Product Info: i.MX6Q/D/S
[INFO]    VPU firmware version: 3.1.1_r46072
[INFO]    VPU library version: 5.4.35
[INFO]    Format: STD_AVC
[INFO]    AVC
[INFO]    Input file "test1.h264" opened.
[INFO]    Output file "test1.yuv" opened.
[INFO]    bitstreamMode 1, chromaInterleave 1, mapType 0, tiled2LinearEnable 0
[INFO]    H.264 Profile: 66 Level: 40 Interlace: 0
[INFO]    sar_width: 65535, sar_height: 65535
[INFO]    Decoder: width = 1920, height = 1088, frameRateRes = 4294967295, frameRateDiv = 4294967295, count = 3
[INFO]    CROP left/top/right/bottom 0 0 1920 1080
[INFO]    1 frames took 162439 microseconds
[INFO]    dec fps = 64.28
[INFO]    total fps= 6.16

Thanks & Regards

Anoop

[igorpadykov]

Hi Anoop

please try Demo Images on i.MX6Q Sabre AI board

http://www.nxp.com/webapp/Download?colCode=IMX6_N7.1.1_1.0.0_ANDROID_DEMO_AI_BSP&appType=license&loc... 

Best regards
igor

[anoopaugustine]

Hi Igor,

Thanks for the quick response!. I have referred mxc_vpu_test application. I would like to know if /dev/fb0 can be configured for YUV color space and I can use this framebuffer directly for H.264 encoding.

Thanks & Regards

Anoop

[igorpadykov]

Hi Anoop

data for encoding may be feed also from framebuffer, one can use

(if necessary) color conversion using examples on

https://community.freescale.com/docs/DOC-94961

Best regards
igor

[anoopaugustine]

Hi Igor,

Thanks for the examples.

I tried to change the VPU clock frequency to MX6_VPU_352M in menuconfig to improve performance. But after setting this the encoder performance is lowered. Is there any other configuration that I need to do for changing VPU clock?

Thanks & Regards

Anoop

[igorpadykov]

Hi Anoop

please check VDD_SOC voltage for 352MHz operation

described in Table 6. Operating Ranges i.MX6DQ Reference Manual
http://cache.nxp.com/files/soft_dev_tools/doc/support_info/iMX6DQPRM.pdf

Best regards
igor

[igorpadykov]

Hi Anoop

please check examples in unit tests (..test/mxc_vpu_test)

imx-test
www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-test-5.7.tar.gz

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------