RGB to YUV for MJPEG and H.264

[gennadiykiryukh]

I was testing IPU + MJPEG encoder and I encountered a small problem.

I took a 2592x1944 bitmap picture and extracted RGB raw data. I ran that data through RGB->YUV in IPU and then to MJPEG to generate one JPEG image. The output image appeared to be dull in contrast (as if looking through dirty glass).

Turns out IPU was not using the right RGB->YUV conversion coefficients. The current driver has generates Y in range of 16 to 235 using the following equations.

Y = (0.257 * R) + (0.504 * G) + (0.098 * B) + 16

Cr = V = (0.439 * R) - (0.368 * G) - (0.071 * B) + 128

Cb = U = -(0.148 * R) - (0.291 * G) + (0.439 * B) + 128

However, JPEG compression expects YUV (YCbCr) values to be in 0-255 range using the following conversion equation:

Y = (0.299 * R) + (0.587 * G) + (0.114 * B)

Cr = (0.5 * R) - (0.418688 * G) - (0.081312 * B) + 128

Cb = -(0.168736 * R) - (0.331264 * G) + (0.5 * B) + 128

Q1: My conclusion is that the currently included IPU driver is not compatible with MJPEG encoder as is and would require modification to use the right registers. Encoder will work but the colors will be incorrect. Is that right?

Q2: I noticed that IPU driver had 3 color spaces defined (ipu_color_space_t in ipu-v3.h): RGB, YUV, YCbCr. However, it appeared to interpret YUV pixel formats (ex. IPU_PIX_FMT_YUV422P) as YCbCr color space and YUV does not appear to be used (format_to_colorspace() function is ipu_common.c). There is even a comment line in ipu_ic.c file that says

/* TODO: Support YUV<->YCbCr conversion? */

What color space in the driver does YUV refer to? What about YCbCr? There seems to be mixing of color spaces because IPU_PIX_FMT_YUV... pixel formats are interpreted as in YCbCr color space instead of YUV.

Q3: What is the requirement for the H.264 encoder?  What range of YUV values does it expect?

Thank you.

[igorpadykov]

Hi Gennady

please look at examples in

https://community.freescale.com/docs/DOC-94961

imx-test
www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-test-5.7.tar.gz

try nxp gstreamer plugins described in attached Linux Guide sect.7.3.3 Video encoding

use nxp bsps on

http://www.nxp.com/products/software-and-tools/software-development-tools/i.mx-software-and-tools/i....

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[gennadiykiryukh]

In my application I will not have a gstreamer. My question was not about how to use IPU or how to use encoder or how to run a demo encoder. It is about the current implementation of IPU driver, supported pixel formats, and what format is needed for MJPEG encoder. What I discovered is that it appears the currently implemented IPU driver does not support proper RGB->YUV conversion that is required from MJPEG. I just want to make sure I did not overlook something.

I was able to test H.264 encoding of YUV data using mxc_vpu_test that was provided. I also made my own program that takes a YUV data for a single 2592x1944 frame and generates JPEG file using MJPEG encoder. JPEG format specifies YCbCr input where each component has value range from 0 to 255. Current implementation of IPU driver only supports YUV(YCbCr) with value ranging from 16 to 235.

I extracted RGB data from this image:

I ran the RGB data through the IPU driver. I specified input pixel format as IPU_PIX_FMT_RGB24 and output format as IPU_PIX_FMT_YUV422P. The output YUV data was fed into MJPEG encoder to generate a JPEG file. This was the output image:

The output image appears faded and the colors appear to be gray-ish. This happened because JPEG requires YUV (YCbCr) values to be in 0-255 range where as IPU driver generates the YUV data in 16 - 235. The lowest black level in the output image ends up at RGB=16,16,16 and the brightest white is RGB=235,235,235.

Did I overlook anything?