This is Android code 13.4.1.04 for imx6 quad core chip. Funtion is _init_csc in kernel_imx/drivers/imx/ipu3. Matrix rgb2rgb_coeff has diagonal elements 0x80 and according to the code scale value is 2. Comparing with the chip spec chapter 22.214.171.124. IC Task Parameter Memory, scale value 2 means that 1/2 is applied to the coefficient. Therefore the final value is (0x80 * 1/2) / 256 = 0x40 / 256 = 0.25. I think that the intent was to apply 1 so the coefficient should be 0x80 but scaling should b be 0 which means that 2 is applied: (0x80 * 2) / 256 = 0x100 / 256 = 1.
Am I missing something?
Also, I'm not clear why are A1 and A2 in rgb2ycbcr_coeff coded as 0x0200 to get 128 as per the formula given in the code. Can you please explain me how we get 128 fro 0x200?