Perfomence convert Bayer (RAW10) to RGB (888 or 565)

sergiiparadiuk · ‎05-31-2022

Hello,

Our company designs a new prodcut.

We selected to use iMX8M-Mini version of the MPU and selected a MIPI-CSI camera.

The camera output has Bayer (RAW10), but we need to compress the data by 1080p 60 frames H.264 ()

Questions:

1) Will it be enough imx8 to convert 1080p 60 frames from Bayer (raw10) to RGB (565 or 888) and compress the data by H.264? (please let me know how much resources will need and maximum frames which could get)

2) Which is the best way to do software converting from raw10 to RGB? (OpenCV lib or something else )

3) If it's not enough imx8 for this, what could you recommend? Maybe put an external ISP chip? (we need a small and cheap solution)

Thank you very much.

sergiiparadiuk · ‎06-02-2022

Thank you very much for the help.

We will go with plus.

malik_cisse · ‎06-01-2022

Hi,

I can recommand NXP imx8m-Plus rather than imx8m-Mini. It has complete ISP with hardware debayer already integrated and there are plenty of ready to go examples using Basler camera modules.

If you need to do debayering in software, I have tested 2 methods:

1-openCV (3840x2160 debayers in ~40ms) which would be enough for 1080p60 but it will probably take some CPU power.

2-NEON based gstreamer implementation from Phytec. You will find it within their imx8mp BSP.

bayer2rgbneon is a gstreamer debayer plugin.

The bayer2rgbneon plugin is in gitlab and is very active:

https://gitlab-ext.sigma-chemnitz.de/ensc/gst-bayer2rgb-neon

The plugin uses the https://gitlab-ext.sigma-chemnitz.de/ensc/bayer2rgb library with NEON acceleration on ARM and ARM64 platforms. According to Phytec It should attein performances arround 1920x1080@28fps.

Neon or "MPE"(Media Processing Engine) is a combined 64- and 128-bit SIMD instruction set that provides standardised acceleration for media and signal processing applications.

Note however that debayering alone is not enough to have nice images. You also need to do further processing:
White balance, Color correction, defect pixel removal, gamma correction etc which are all included in imx8mp hardware ISP.

There is also Software ISP based on openCL from NXP: (https://www.nxp.com/design/designs/i-mx8-software-image-signal-processing:SOFTISP-I.MX8)

Udhay_ · ‎12-22-2022

Have you tested bayer2rgb with raw10?