Raw Camera with ISP in i.MX8MPlus

[jsYeom]

We are planning to use samsung S5K4HA camera sensor which is RAW10 sensor. We are currently able to receive RAW images without ISP. (in/out BG10) (We simply ported the camera driver. media/i2c/newcamera.c) I'd like to use ISP in i.MX8MPlus, which is used in conjunction with camera. Can you please help what changes are required to use ISP with some example? I found another similar question in community and I saw you guide the isp-imx-4.2.2.6.0.bin package and the isp-vvcam sample code. I don't think there is enough information to port camera. I need detailed guidelines for porting a new camera with isp.

[joanxie]

currently the detailed Sensor Porting Guide hasn't been released, still under review phase, I can sent to you a simply steps for porting new sensor which includes ISP

1)Define sensor attributes and creating sensor instance.

2)Defining the camera driver configuration data structure.

3)Setting the sensor #defines. For example:

a)SENSOR_SLAVE_ADDR

b)SENSOR_MAX_GAIN_AEC

c)SENSOR_PIX_CLOCK

d)SENSOR_MIPI_LANES

4)Writing customized exposure parameters.

5)Exposure function is unique for each sensor.

6)Refer data sheet of the sensor for customized calculation of exposure parameters.

7)Modifying the sensor driver.

8)To specify a camera sensor, the sensor driver must be modified.

9)The data structure is defined in file vvcam/V4l2/sensor/<sensor_vendor>_<sensor>/<sensor>.c

10)Set up HDR.

11)Enable the HDR function of the ISP.

12)Define MIPI lanes.

13)Update the MIPI lanes.

For example, in file ../isi/drv/OV2775/source/OV2775.c

  pIsiSensorCaps->MipiLanes = ISI_MIPI_4LANES

14)Sensor compand curve.

15)Sensor white balance.

 a)ISP AWB is used in normal mode.

b)In native HDR mode, black level and white balance calibration should be done before the image synthesis at the sensor.

[jsYeom]

I'll ask you a few more questions. Which part should I modify 1 through 6? Do I just need to add sensor driver in vvcam? Or Should I merge isp-imx-4.2.2.6.0 and vvcam to work?

[joanxie]

to be honest, this is very complex steps, I couldn't share all of them in the public community since we haven't released the senor poring guide, this document is still under review, I couldn't attach it here, if you need detailed step by step and which structure and file you need change, pls wait for the official released document

[kiseok_ko]

@joanxie 

>> to be honest, this is very complex steps, I couldn't share all of them in the public community since we haven't released the senor poring guide, this document is still under review, I couldn't attach it here, if you need detailed step by step and which structure and file you need change, pls wait for the official released document

I also need same solution(for imx274), but Information is not enought.

Can I get "official release document"?

I could not find mentioned documnet, Not yet.

[khang_letruong]

Hi @kiseok_ko 

I think that you can find the necessary document in the release note package for iMX (tab Documentation --> Release Note) :

After downloading extracting the selected version of iMX Linux release, you will find a file name iMX8MP_CAMERA_DISPLAY_GUIDE.pdf :

Regards,

K.

[kiseok_ko]

Thank you, khang_letruong.

I got stream&capture on ISP with imx274.

but, color and image quality is not good.

we got stream&capture on ISI with imx274.

Two images are different quality.

I think this is because of xml data.

but xml description is not included in any other doc.

Could you help me, one more time?

B.R.

kiseokko

[khang_letruong]

Hi @kiseok_ko ,

What do you get as image format behind ISI?  And how do you compare with output behind ISP? Should they be converted to same format?

For XML documentation, NXP recently released AN13565 Sensor Calibration Parameter Specifications for ISP Calibration Tool XML Generator. Please refer to https://community.nxp.com/t5/i-MX-Processors/imx8mp-ISP-Calibration-Tool-XML-Generator/m-p/1442320 for more detail.

Regards,

K.

[kiseok_ko]

First, Thank you @khang_letruong

You answers are very helpful to me.

============================

But, I cannot solved this problems.

============================

Dear All.

I can not handle ISP parameters, dynamically.

Actually, We need below pipelines. Is it possible?

1. sensor -(bayer)-> mipi -(bayer)-> isp0 -(bayer)-> captured(memory)

     : do not need dwe.  auto or manual control exposure & gain.

     : ISP 4k capture with ISI capture (13.4 Image Sensing Interface (ISI) module , IMX8MPRM.pdf p5265)

2. [sensor, with isp] -(yuv422)-> mipi -(yuv422)->isp1-(yuv422) -> captured(memory) : need dwe. optionally.

   ;  13.14.2.2.2 MIPI Inputs (IMX8MPRM.pdf p5899)
The ISP can support the following image data formats through the MIPI interface (see the
MIPI-CSI2 standard for details about the image data formats and byte sequences):
• YUV 4:2:2 8bit / 10bit (to YUV processing chain)

who could help me?

Thanks a lot.

Best Regards.

kiseokko.