i.MX8MP MIPI_CSI2 Interface Limitations

[brunomellotx]

Hello,

I want to better understand the limitations of the MIPI_CSI2 interface on the i.MX8MP. As a customer using one of our SoMs featuring this SoC is running into some issues.

The i.MX8MP features two MIPI-CSI interfaces. Reading the Datasheet and the Reference Manual the only limitations mentioned are related to the pixel clock:

For single Camera, MIPI CSI 1 can support up to 400/500 MHz pixel clock in the
Nominal/Overdrive mode.
• For single Camera, MIPI CSI 2 can support up to 277 MHz pixel clock.
• For dual Camera, both MIPI CSI can support up to 266 MHz pixel clock.

We are running into some issues with MIPI CSI 2 with higher resolutions (> 2048 horizontal). From my research this is likely not a limitation of the MIPI CSI interface, but with the ISI.

On the ISI side, I see some limitations of resolutions with a horizontal resolution higher than 2048, specifically on the reference manual, section 13.4 Image Sensing Interface (ISI).

There, it mentions that the line buffers need to be shared to achieve higher resolutions.

It is my understanding that this should not limit the MIPI CSI 2 interface of using higher resolutions, as the pixel link crossbar should allow the data from MIPI CSI 2 to be routed to a channel that has support for chain buffering.

Is this understanding correct? If so, how could this be configured on the device tree when running linux?

Best Regards,

Bruno

[brunomellotx]

To clarify the question here: It is about the second MIPI CSI interface (MIPI_CSI2) of the i.MX8MP.

Higher resolution cameras work without issues with the first MIPI CSI interface (MIPI_CSI1).

[Bio_TICFSL]

Hello,

Yes you understanding are correct!

Regards

[brunomellotx]

Hello @Bio_TICFSL,

Thanks for the confirmation.

How can I configure the device tree to connect MIPI_CSI2 to an ISI channel that supports horizontal resolutions higher than 2048?

For my testing I use this overlay with an OV5640 camera, but if you have an example from NXP's BSP or for another sensor that would already be very useful.

Best Regards,

Bruno

[Bio_TICFSL]

Hello,

By default, the imx8mp-evk.dts supports single ov5640 camera via CSI0 interface:

- ov5640 sensor node :

	ov5640_0: ov5640_mipi@3c {
		compatible = "ovti,ov5640";
		reg = <0x3c>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_csi0_pwn>, <&pinctrl_csi0_rst>, <&pinctrl_csi_mclk>;
		clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
		clock-names = "xclk";
		assigned-clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
		assigned-clock-parents = <&clk IMX8MP_CLK_24M>;
		assigned-clock-rates = <24000000>;
		csi_id = <0>;
		powerdown-gpios = <&gpio2 11 GPIO_ACTIVE_HIGH>;
		reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
		mclk = <24000000>;
		mclk_source = <0>;
		mipi_csi;
		status = "okay";

		port {
			ov5640_mipi_0_ep: endpoint {
				remote-endpoint = <&mipi_csi0_ep>;
				data-lanes = <1 2>;
				clock-lanes = <0>;
			};
		};
	};

- mipi-csi2 node :

&mipi_csi_0 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	port@0 {
		reg = <0>;
		mipi_csi0_ep: endpoint {
			remote-endpoint = <&ov5640_mipi_0_ep>;
			data-lanes = <2>;
			csis-hs-settle = <13>;
			csis-clk-settle = <2>;
			csis-wclk;
		};
	};
};

You should have the ov5640 camera module(s) connected to the MIPI-CSI2 connectors of the board (EVK for example) so that the driver can communicate with and identify it before registering into the system as /dev/video0.

If you expect to use dual ov5640 sensors (via both mipi-csi2 interfaces), work-around might be necessary on power-down and reset signals of second one : https://community.nxp.com/t5/i-MX-Processors/Dual-OV5640-cameras-on-iMX8M-Plus/m-p/1250820

Regards

There's also other examples of using dual camera sensors such as imx8mp-evk-dual-ov2775.dts or imx8mp-evk-dual-basler.dts. Please have a look.

[brunomellotx]

Hello @Bio_TICFSL,

Thanks for your response and sorry for the delay here.

Unfortunately, the examples you sent do not give me a working example of the second MIPI CSI interface being used for a higher resolution image.

When I use the following configuration, v4l-ctl does not show a mode with a resolution higher than 1920x1080.

&i2c2 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	camera@3c {
		compatible = "ovti,ov5640";
		reg = <0x3c>;
		clocks = <&clk_camera_mezzanine_csi>;
		clock-names = "xclk";
		AVDD-supply = <&regulator_camera_mezzanine>;
		DVDD-supply = <&regulator_camera_mezzanine>;
		DOVDD-supply = <&regulator_camera_mezzanine>;
		PVDD-supply = <&regulator_camera_mezzanine>;
		/* Verdin GPIO2 (JP6 1-2) - Mezannine Camera Connector 22 */
		powerdown-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
		/* Verdin GPIO1 (JP5 1-2) - Mezannine Camera Connector 11 */
		reset-gpios = <&gpio1 0 GPIO_ACTIVE_LOW>;

		port {
			ov5640_mipi_1_ep: endpoint {
				clock-lanes = <0>;
				data-lanes = <1 2>;
				remote-endpoint = <&mipi_csi1_ep>;
			};
		};
	};
};

&isi_1 {
	status = "okay";

	cap_device {
		status = "okay";
	};
};

&mipi_csi_1 {
	#address-cells = <1>;
	#size-cells = <0>;
	status = "okay";

	port@1 {
		reg = <1>;

		mipi_csi1_ep: endpoint {
			csis-hs-settle = <13>;
			csis-clk-settle = <2>;
			csis-wclk;
			data-lanes = <2>;
			remote-endpoint = <&ov5640_mipi_1_ep>;
		};
	};
};

Eventually this will be for a dual-camera use case, but for now I just want to be able to capture an image with a larger resolution from the second MIPI-CSI interface of the i.MX8MP.
The use case is for sporadic captures, so the cameras could use the same ISI processing pipeline, just not at the same time.

It seems that MIPI_CSI2 is hard-coded somewhere to always use the second ISI processing pipeline (isi_1 node in the device tree), which cannot use both line buffers.

How can I configure it so the second MIPI-CSI interface (mipi_csi_1 node) is connected to the first ISI processing pipeline (isi_0 node)?

Best Regards,

Bruno

[Bio_TICFSL]

Hello,

Please check it:

https://www.nxp.com/docs/en/application-note/AN13430.pdf

Regards

[brunomellotx]

Hello @Bio_TICFSL,

Thanks for pointing to this Application Notes.

I had read it in the past but missed the table with the ISI input assignemnts.

For future reference, if someone needs to use a higher resolution camera with the second MIPI_CSI interface of the i.MX8MP, you need to set the following in the device tree:

&isi_0 {
	interface =  0 2>;
};

From my testing the intended use case of switching this at runtime has proven to be unsupported by the drivers involved.

Can I use the ISP for 2x high resolution (more than 2048 width) captures? Or are there similar limitations there?

Best Regards,

Bruno

[Bio_TICFSL]

Hello,

You can use one at 4K, and two at 2048.

Regards

[brunomellotx]

Hello @Bio_TICFSL,

> You can use one at 4K, and two at 2048.

I understand this is a limitation of the ISI.

Is this also the case if we use the ISP and not the ISI?

Or is this not possible?

Best Regards,

Bruno

[Bio_TICFSL]

Hello.

Yes it applies to both.

Regards

[brunomellotx]

Hello @Bio_TICFSL,

Thanks for the clarification.

This topic can be closed.

Best Regards,

Bruno