[iMX8MP] ISP Camera Porting Guide for MIPI 1-lane sensor

[khang_letruong]

Dear NXP Team,

In Chapter 6 Camera Sensor Driver in V4L2 Mode of IMX8MPCSPUG.pdf, one of the step for porting the camera sensor is :

5. Define MIPI lanes

However, in isp-imx-4.2.2.6.0/isp-imx-4.2.2.6.1/isp-imx-4.2.2.11.0, there's only following definition :

units/isi/include/isi_common.h
71:#define ISI_MIPI_LANES                      0x00000060
108:#define ISI_MIPI_OFF                        0x80000000      //!< MIPI is disabled
109:#define ISI_MIPI_4LANES                     0x00000004
110:#define ISI_MIPI_2LANES                     0x00000002

1. How do I configure the number of MIPI lane(s) for my sensor correctly knowing that our sensor support only 1-lane MIPI output, please?

2. By the way, in page 5 of IMX8MPCSPUG.pdf, it is said :

— Kernel Working Mode: VVCAM has two types of working modes in the kernel:
1. V4L2 Mode: kernel driver that acts as a part of V4L2 kernel driver, register device name and operations as a V4L2
sub-device style. This mode is compatible with the V4L2 sensor device format.

I wonder what would be the second mode, please ?

Best regards,
Khang

[malik_cisse]

Hi Khang,

I am struggling with the same issues as you trying to make the ISP work with custom sensor.

Did you come up with the minimal steps to do so?

Your input will be much appreciated.

Best regards, Malik

[khang_letruong]

Dear @igorpadykov ,

Thanks for your quick reply and pointing out, but I'm already aware of how to configure data-lanes/clock-lanes in dts for standard MIPI-CSI camera porting. However, I would like how to do that correctly in the context of iMX8MP's ISP (within isp-imx framework extracted from isp-imx-<version>.bin) based on NXP's Camera Porting Guide using ISP, please!

Best regards,

Khang

[igorpadykov]

ISP does not use lanes as it receives data from MIPI-CSI. MIPI-CSI uses lanes.

Best regards
igor

[khang_letruong]

Hi @igorpadykov ,

Meanwhile, I found this in Chapter 6.2.5 Define MIPI Lanes of iMX 8M Plus Camera Porting Guide (IMX8MPCSPUG.pdf) :
Screenshot from 2021-05-07 00-34-54.png

And all stuffs starting with isi, Isi or ISI belong to ISI Layer which is part of the ISP Software Architecture. Could you explain more, please?

Best regards,

Khang

[igorpadykov]

>And all stuffs starting with isi, Isi or ISI belong to ISI Layer which is part of the ISP Software Architecture.

right, but ISI and ISP are different modules. ISP does not use lanes.

1.jpg

 

Best regards
igor

[khang_letruong]

Dear @igorpadykov ,

I'm afraid that you are referring the ISI which is the Image Sensing Interface hardware block/module while I'm mentioning the ISI layer which is Independent Sensor Interface layer in the ISP Software Architecture as below :
Independence-Sensor-Interface.png

In term of software implementation, one is in kernel-space of linux-imx :

drivers/staging/media/imx/imx8-isi-hw.h
drivers/staging/media/imx/imx8-isi-hw.c
drivers/staging/media/imx/imx8-isi-core.h
drivers/staging/media/imx/imx8-isi-cap.c
drivers/staging/media/imx/imx8-isi-m2m.c
drivers/staging/media/imx/imx8-isi-core.c

and other is in user-space (!?!?) within the isp-imx :

units/isi
units/isi/include_priv/isi_priv.h
units/isi/source/isi.c
units/isi/source/isisup.c
units/isi/include/isi_common.h
units/isi/include/isi_iss.h
units/isi/include/isi.h

I find this ISI a little bit confusing but I don't see that they are relevant each other. Do you agree ?

Just let you know that we were able to fetch data from our 1 lane MIPI-CSI camera sensor to the Image Sensing Interface module then made it appear as standard V4L2 video device (/dev/videoX) by bypassing the ISP block / module. Now with the official release and support of ISP related documentation and source code, we wish to utilize the ISP for advanced processing.

Best regards,

Khang

[igorpadykov]

I asked internally, answer below:

--------------------

There are not changes to need for 1 lanes sensor at vvcam part(ISP).

1. Sensor driver support 1 lanes.

2. CSI2 driver(imx8-mipi-csi2-sam.c) change to 1lanes at dtb : 

       of_property_read_u32(node, "data-lanes", &state->num_lanes);

--------------------

Best regards
igor

[khang_letruong]

Dear @igorpadykov ,

I would like to re-launch this question since I'm in the context of ISP-IMX and there's certain definition of number of lanes within units/isi/include/isi_common.h :

#define ISI_MIPI_OFF                        0x80000000      //!< MIPI is disabled
#define ISI_MIPI_4LANES                     0x00000004
#define ISI_MIPI_2LANES                     0x00000002

...

#define ISI_MIPI_LANES                      0x00000060

Which then are used in each sensor ISP based driver, for example :

pIsiSensorCaps->MipiLanes = ISI_MIPI_4LANES;

And escalated to units/isi/source/isi.c :

    struct vvcam_csi_lane_cfg csi_lane_cfg;
    csi_lane_cfg.mipi_lane_num = Caps.MipiLanes;
    result = ioctl(pHalCtx->csi_fd,VVCSI_IOC_S_LANE_CFG,&csi_lane_cfg);
    if (result != 0)
    {
        TRACE( ISI_ERROR, "%s: Set csi mipi lane Error\n", __func__);
        return ( RET_FAILURE );
    }

Also, I have confusion of the relationship of the number of lanes defined in above units/isi/include/isi_common.h and ones defined in device-tree as your suggestion, for example: 

imx296_0: imx296_mipi@1a {
    compatible = "sony,imx296_mipi_vvcam";
    ...
    port {
        imx296_mipi_0_ep: endpoint {
            data-lanes = <1>;
            clock-lanes = <0>;

            remote-endpoint = <&mipi_csi0_ep>;
        };
    };
};

Are they both programmed in the following register ?
MIPI_CSIx_CSIS_COMMON_CTRL.png

Last thing, since 1-lane definition is missing in above units/isi/include/isi_common.h, I plan to add it as following :

#define ISI_MIPI_OFF                        0x80000000      //!< MIPI is disabled
#define ISI_MIPI_4LANES                     0x00000004
#define ISI_MIPI_2LANES                     0x00000002
#define ISI_MIPI_1LANE                      0x00000001      // My definition 

What do you think ?

Thanks in advance,
Khang.

[igorpadykov]

Hi Khang

one can look at dts "data-lanes", "clock-lanes" :

https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mp-evk-b...

https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mp-evk.d...

Best regards
igor