Colour inversion issue in Android while streaming via skype!! - imx6dl controller + ov2685 camera

[navinars]

Hi,

   I am working on an android device using imx6dl as the microcontroller. I have interfaced an omnivision camera ov2685 with the board. The camera preview is correct, and image capture as well as video capture also works fine using the camera app in the board. The issue is when I call the device from my mobile, pc etc via video calling applications like Skype, Linphone etc, the colors are swapped (The u and v parameters are interchanged).

The camera provides a YUV422 (yuyv) stream to the ipu. This stream is converted to NV12(YUV420) in android and is displayed on the lcd (after conversion to RGB), and while capturing the image, NV12 is compressed and saved in JPEG format. 

Basically, Android uses NV21(YVU420SP) format or YV12(YVU420P) format. But since the imx6dl microcontroller does not support NV21 format, as far as I understand, the freescale Android source code has been modified for NV12. And it does work, when I am opening the camera or accessing it using any other apps in the device only. But when I stream the camera output via skype, then on the other end, the image is inverted. (Essentially, the NV12 image manifests itself as NV21 in the other end). 

 I am assuming that this issue arises because when the NV12 image is streamed through the linphone/skype app, it gets converted to NV21 which is why this inversion occurs.

I verified that the image viewed in the mobile is the inverted one by modifying the NV12 image to YV12 in the device and displaying the inverted image. This one was the same image as seen in the mobile when the NV12 image was streamed. 

Q1. Is there a way to reformat the video stream which is sent via skype? I have been unable to figure out where exactly in android this change should take place (still working on it). A simple interchanging of u and v parameters should do the trick, but the changes I tried to make so far has not had any effect till date.

Q2. Have anyone noticed this issue before? I have not been able to find anyone else who reported this problem. 

Please let me know if you guys have any idea regarding this. I have attached three images below. 

"1.correct_image_in_camera_app.png" -> This is the correct pattern. The image was taken from Camera app in device. 

"2.imx_device_linphone_image.png" -> This is the linphone app in the imx device. The small pattern shown in the low right corner is the correct image also. 

"3.mobile_linphone_stream.jpg" -> This is the linphone app in my smartphone. If you view the images, you can see that the yellow/aqua colors and the red/blue colors seems to be interchanged, as well as a difference in the color tones. This is because the u and v components of the stream from the camera has been swapped. 

stathisv‌  SaurabhPatel‌   igorpadykov‌  jamesbone‌ GaryBisson‌ wangshengjiu‌ GaoJianzhong‌ YuriMuhin_ng‌ wallyyeh‌

[igorpadykov]

Hi Navinar

seems this could be debugged where color conversion happens sending

some data patterns from memory, not using camera. As streaming via skype is

not supported in official nxp bsps, one can apply to NXP Professional Services

to get more support:

http://www.nxp.com/support/nxp-professional-services:PROFESSIONAL-SERVICE

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[navinars · Accepted solution]

Hi Igor,

    I managed to solve the issue, but used a different method to do it. I converted the format in Android layer from NV12 to YU12(Android actually uses YV12 but YU12 is implemented in imx android bsp). The original issue was still present, i.e., same colour inversion while streaming was happening. So I switched the u and v addresses in idmac so that the stream was output in YV12 format even though the format mentioned was YU12. This corrected the colour issue and I was able to stream the video successfully.

At this point I faced another issue. In freescale Android Source code, when capturing the image, the jpeg compression is implemented only for YUV420SP (NV12) or YUV422I formats. To solve the problem, I modified the captured image buffer which was originally in YUV420P format (YYYYYYYYU1U2V1V2) to YUV420SP format (YYYYYYYYU1V1U2V2) format before passing it on for jpeg compression. This took care of the image capture problem also.

As of now, everything is working fine. I would have liked to solve this problem in the Android Source code itself using the NV12 format, but I was not able to gain much progress in that area. Getting professional support from NXP is pretty expensive so it is not an option. :smileygrin:  Do let me know if you come across some information regarding converting NV12 to NV21 in the android BSP. Thanks again for your help!! 

Best Regards, 

Navinar