Capturing YUV420 from MIPI CSI camera for VPU compression

[ofer_livny]

Hi guys,

I'm using the iMX6Q MIPI CSI and IPU to capture video from a camera.

My camera supports multiple data types, and I am able to capture with the data-types: RGB and YUV422.

I am now experimenting with YUV420, and I hope I'll be able to send the video stream into the VPU for compression (Which only supports YUV420 for most codecs).

Well, it is not clear by the iMX6 and IPU spec if I can use this YUV420 datatype:

According to IMX6DQRM.pdf I found that the MIPI component supports all formats, CSI2IPU component supports all format, but it seems that the IPU does not support YUV420 (it is missing from the description of register IPUx_CSI0_SENS_CONF)

If I understand correctly, I can only use YUV420 if I configure it as a GENERIC MIPI data, and I won't be able to have the IPU working on it (i.e. stream the frame through its conversion block, or format converter in the IDMAC)

However, since the VPU codecs can only work on YUV420, is it possible to create a capture loop which compresses the video without using CPU for format conversion ?

Please advice :smileyhappy:

Ofer

[ofer_livny]

As far as I know you can use the IC for conversion.

The only problem is that its output is limited to 1024x1024, which means that higher resolution cannot be encoded.

Saying that, I must add again that this means that the VPU encoders are very VERY limited:

The VPU encoders support 1080p (1920×1080) but only with a specific variation of YUV420 (Y channel must be planar)

The only way to encode 1080p video will be to use a camera sensor which streams this exact variation of YUV420.

Since this PLANAR format is not exactly the best known way for streaming video (and thats an understatement) - it actually means that VPU encoders can effectively work only on 1024x1024 resolution. (which is the IPU's output limit).

This means more complicated software, heavier RAM usage (capture interleaved YUV420 image to memory, read into IPU IC, write planar YUV420 to memory, and read it into VPU...), and resolution limit lower than 1080p.

Hopefully someone at Freescale will explain what this 1080p encoding capability can be really used for...

For now, I suppose I'm stuck with YUV422 and MJPEG encoding. Not too bad, but not optimal.

[fcs]

720p is enough for us.

You say you are "stuck with YUV422 and MJPEG encoding", does this use the VPU?

[ofer_livny]

Yeah, I use the VPU to encode JPEGs. The MJPEG codec accepts many variations of YUV, and is not limited to YUV420 as the reset of the VPU codecs.

[maxj]

Do you use the vpu_lib API for your JPEG encoding?  If so, how do you determine values to use for the following arrays in the EncMjpgParam structure?

Uint8 huffVal[4][162];

Uint8 huffBits[4][256];

Uint8 qMatTab[4][64];

Uint8 cInfoTab[4][6];

I saw these were additions to the documentation I have ("i.MX 6Dual/Quad VPU Application Programming Interface Linux Reference Manual" dated 8/2011... haven't been able to find anything more recent), but the are no explanations for what they are.  I can guess what huffVal[0], huffVal[1], huffVal[2], qMatTab[0], qMatTab[1], and qMatTab[2] are, based on the JPEG specs and other descriptions in the document.  Perhaps computation of huffBits[][] can be found elsewhere in further JPEG spec research.  However, cInfoTab[][] looks like it will require vpu_lib reverse-engineering.

Either way, could you provide some sample code?

[Yuri]

huffVal[4][162] - A list of the 8-bit symbol values in Huffman tables.

huffBits[4][256] - A 16-byte list giving number of codes for each code length from 1 to 16 in Huffman tables.

qMatTab[4][64] - Quantization tables

cInfoTab[4][6] - Component information tables.