topic Re: iMX8M MIPI-CSI 4-lane configuration in i.MX Processors

iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Thu, 21 Mar 2019 01:01:30 GMT

Hello,

I came across an odd behavior of the iMX8M processor's MIPI-CSI camera input.

I have an AR0330 imager, which I wrote a driver for, and is working pretty neat.

The problem is, it only works if I use it with a single MIPI-CSI lane.

The moment I try to make it work with 1+ lanes (2 or 4), the image that gets in looses all the data on every other lane except "Lane 0" (at least that's what it looks like what's happening).

In the working configuration, my .dtsi looks like the following (the relevant parts):

csi1_bridge: csi1_bridge@30a90000 {

compatible = "fsl,imx8mq-csi", "fsl,imx6s-csi";

reg = <0x0 0x30a90000 0x0 0x10000>;

interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>;

clocks = <&clk IMX8MQ_CLK_DUMMY>,

<&clk IMX8MQ_CLK_CSI1_ROOT>,

<&clk IMX8MQ_CLK_DUMMY>;

clock-names = "disp-axi", "csi_mclk", "disp_dcic";

dma-coherent;

status = "disabled";

};

mipi_csi_1: mipi_csi1@30a70000 {

compatible = "fsl,mxc-mipi-csi2_yav";

reg = <0x0 0x30a70000 0x0 0x1000>; /* MIPI CSI1 Controller base addr */

interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;

clocks = <&clk IMX8MQ_CLK_DUMMY>,

<&clk IMX8MQ_CLK_CSI1_CORE_DIV>,

<&clk IMX8MQ_CLK_CSI1_ESC_DIV>,

<&clk IMX8MQ_CLK_CSI1_PHY_REF_DIV>;

clock-names = "clk_apb", "clk_core", "clk_esc", "clk_pxl";

assigned-clocks = <&clk IMX8MQ_CLK_CSI1_CORE_DIV>,

<&clk IMX8MQ_CLK_CSI1_PHY_REF_DIV>,

<&clk IMX8MQ_CLK_CSI1_ESC_DIV>;

assigned-clock-rates = <133000000>, <100000000>, <66000000>;

power-domains = <&mipi_csi1_pd>;

csis-phy-reset = <&src 0x4c 7>;

phy-gpr = <&gpr 0x88>;

status = "disabled";

};

ar0330_mipi: ar0330_mipi@10 {

status = "okay";

compatible = "ar0330";

reg = <0x10>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_csi1>;

clocks = <&clk IMX8MQ_CLK_CLKO2_DIV>;

clock-names = "csi_mclk";

csi_id = <0>;

rst-gpios = <&gpio5 28 GPIO_ACTIVE_HIGH>;

mclk = <24000000>;

mclk_source = <0>;

port {

ar0330_mipi1_ep: endpoint {

remote-endpoint = <&mipi1_sensor_ep>;

};

&csi1_bridge {

fsl,mipi-mode;

fsl,two-8bit-sensor-mode;

status = "okay";

port {

csi1_ep: endpoint {

remote-endpoint = <&csi1_mipi_ep>;

};

&mipi_csi_1 {

#address-cells = <1>;

#size-cells = <0>;

status = "okay";

port {

mipi1_sensor_ep: endpoint1 {

remote-endpoint = <&ar0330_mipi1_ep>;

data-lanes = <1>;

};

csi1_mipi_ep: endpoint2 {

remote-endpoint = <&csi1_ep>;

};

This gets me the image, and I can switch between RAW8 and RAW12 image format after patching the mx6s_capture.c code, with the necessary V4L2 bits and pieces and toggling fsl,two-8bit-sensor-mode;. Everything works like a charm.

Now, when I switch the camera to 4-lane or 2-lane MIPI operation, I modify the following part of the device tree:

port {

mipi1_sensor_ep: endpoint1 {

remote-endpoint = <&ar0330_mipi1_ep>;

data-lanes = <1 2 3 4>;

};

And obviously, my driver's register settings towards the camera, so that end also enables the 4-lane transmission.

At this point, I would expect the interface to work, but I might be missing something.

After I deploy my new device tree, I see the target booting with the new configuration:

[ 1.710335] CSI: Registered sensor subdevice: mxc-mipi-csi2.0
[ 1.714820] mxc-mipi-csi2_yav 30a70000.mipi_csi: Remote device at /mipi_csi1@30a70000/port/endpoint1 XXX found
[ 1.723539] mxc-mipi-csi2_yav 30a70000.mipi_csi1: lanes: 4, name: mxc-mipi-csi2.0
[ 3.999571] mxc-mipi-csi2_yav 30a70000.mipi_csi: Registered sensor subdevice: ar0330 3-0010

I have verified the signals with an oscilloscope and they work as expected, so the data definitely gets up until the device's pads, but unfortunately they seem to stop right there.

Also verified if the polarities are right, but that's also fine. + is + and - is - , so at this point I'd rule out HW problems.

The PCB is simulated using CST-Studio, hence I'd also doubt signal integrity problems, more so because a single lane with the same PCB can receive 4x the datarate, so I'd expect if I have the data rate's 1/4th on the other lanes, routed extremely similarly, that must not be a problem.

I'm not entirely sure where to go for documentation as well, because it seems like the iMX8M Reference Manual has almost zero information on the CSI interface, and there are only 1-2 replies in forums saying that this device is somewhat of a mix of the iMX7 and iMX6 interfaces, regarding MIPI and CSI (?!)

Please let me know if I'm missing any documentation or driver.

Right now I'm using Yocto with kernel 4.9.51-imx_4.9.51, but I checked and none of these drivers received any update in 4.14.78.

There seems to be this reference to an upcoming driver in the freescale .dtsi file:

compatible = "fsl,imx8mq-csi", "fsl,imx6s-csi";

fs, imx8mq-csi -> Is this something that's available somewhere or is it in the makes? Maybe that's what'll be needed for more lanes to work?

Thank you in advance for your answers!

Krisztian Bakos

Re: iMX8M MIPI-CSI 4-lane configuration

julien_jayat — Thu, 21 Mar 2019 09:24:11 GMT

Hi Krisztian,

1) Documentation about the CSI capture is missing from the current version of the IMX8M Reference Manuel.

In the RM for imx6sll, (which is using the same IP and linux driver) the following chapter describes the IP.

"16.4.1 Data Transfer with the Embedded DMA Controllers".

The documentation of the MIPI_DSI interface is in the 8M RM,

see "13.6.4.1.1 CSI2RX Memory Map" for description of the register configuring the number of data line.

2) There are some changed in 4.14.78 in for using "fsl,imx8mq-csi", see commit:

linux-imx - i.MX Linux kernel

But it is not related to the MIPI CSI lines, but improving the stability under high DDR bandwidth conditions.

3) About your problem MIPI CSI lines, you seem to have correctly configured the number of line.

Since you are spreading the same amount of data on more lined can you verify that the frequency on each line in within range?

MIPI CSI : "Supports high speed mode(80Mbps - 1.5Gbps) per lane"

The BSP is delivered with the example of the OV5640 camera using 2 MIPI CSI lines.

I have a question about:

The moment I try to make it work with 1+ lanes (2 or 4), the image that gets in looses all the data on every other lane except "Lane 0" (at least that's what it looks like what's happening).

What do you see exactly? do you have an partial image captured?

Can you make sure that you are not using MIPI CSI Virtual Channel?

Julien,

Re: iMX8M MIPI-CSI 4-lane configuration

igorpadykov — Thu, 21 Mar 2019 11:33:23 GMT

Hi Krisztian

one can try to debug it (pay attention to rxhs-settle calculate,mipi_csi2_parse_dt)

linux/drivers/media/platform/imx8/mxc-mipi-csi2.c

mxc-mipi-csi2.c\imx8\platform\media\drivers - linux-imx - i.MX Linux kernel

MX8M RXHS_SETTLE

Explenation for HS-SETTLE parameter in MIPI CSI D-PHY registers

Best regards
igor

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Thu, 21 Mar 2019 13:15:12 GMT

Thank you for your reply.

To your question "What do you see exactly? do you have an partial image captured? "

It's correct, when I set 2 lanes -> Half the pixels come in

If I set 4 lanes -> I get 1/4 of the image

"Can you make sure that you are not using MIPI CSI Virtual Channel?"

How would I do that? The sensor doesn't support configurable virtual channels. Do you mean to set it on the iMX side?

I will check your suggestion on the lane data rate configuration.

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Thu, 21 Mar 2019 13:17:02 GMT

Very helpful links! Thank you! I will check out these settings.

Re: iMX8M MIPI-CSI 4-lane configuration

julien_jayat — Thu, 21 Mar 2019 16:53:31 GMT

I was asking about the sensor part.

You answered it. IMX8M doesn't support the Virtual Channel.

For 2 lines configuration:

Do you see

1. continuous pixel/ missing pixel

2. valid top half of the frame and missing bottom half

1. would make me think of problem in the capture side.

2. would make me think of configuration problem between the MIPI CSI Rx and the CSI capture engine.

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Mon, 02 Nov 2020 14:18:55 GMT

Update:

Here is the image I get, with the shown resolution (2304x1536 - RAW8) with 2 lane configuration. I set up the sensor to output on 2 lanes, as that way the MIPI clock is in range for the CSI2 receiver (83.4 MHz).

igorpadykov‌
To your point above, I tried to use the driver "fsl,mxc-mipi-csi2", however, that dies on loading saying it can't find

Failed to get mipi CSR register

So I assume that version of the driver is incompatible with the MIPI-CSI2 receiver in the iMX8M.

I switched back to the

compatible = "fsl,mxc-mipi-csi2_yav";

However, there the hs-settle devicetree parameter gets ignored for whatever reason. It seems to be hard-coded, so I modified the following in 'mxc-mipi-csi2_yav.c':

#define GPR_CSI2_1_S_PRG_RXHS_SETTLE(x) (((x) & 0x3F) << 2)

//Hard-code HS_SETTLE to 0x01 -> 80-90MHz pixel clock

static u8 rxhs_settle[2] = { 0x01, 0x01 };

regmap_update_bits(csi2dev->phy_gpr.gpr,

csi2dev->phy_gpr.req_src,

0x3FFF,

GPR_CSI2_1_RX_ENABLE |

GPR_CSI2_1_VID_INTFC_ENB |

GPR_CSI2_1_HSEL |

GPR_CSI2_1_CONT_CLK_MODE |

GPR_CSI2_1_S_PRG_RXHS_SETTLE(csi2dev->

hs_settle));

(I modified only the values, the other things I just copied here to show where it gets used)

My current .dtsi file looks like this (even though I think those values get ignored in the driver)

mipi1_sensor_ep: endpoint1 {

remote-endpoint = <&ar0330_mipi1_ep>;

data-lanes = <1 2>;

csis-hs-settle = <1>;

csis-clk-settle = <0>;

//csis-wclk;

};

And it comes up with:

root@b0sample:~# dmesg | grep csi
[ 1.703847] mxc-mipi-csi2_yav 30a70000.mipi_csi1: mipi_csi2_probe
[ 1.708724] CSI: Registered sensor subdevice: mxc-mipi-csi2.0
[ 1.713215] mxc-mipi-csi2_yav 30a70000.mipi_csi: Remote device at /mipi_csi1@30a70000/port/endpoint1 XXX found
[ 1.721934] mxc-mipi-csi2_yav 30a70000.mipi_csi1: lanes: 2, name: mxc-mipi-csi2.0
[ 3.904325] mxc-mipi-csi2_yav 30a70000.mipi_csi: Registered sensor subdevice: ar0330 3-0010

I did not mention yet,but the command I use to capture the data is:

v4l2-ctl --set-fmt-video=width=2304,height=1536,pixelformat='BA81' --stream-count=1 --stream-to=MIPIimage.bayer --stream-mmap=5

The allocated video buffer is correct, and the file size is as expected.

I have attached the raw incoming data as well, if you want to take a look what's coming in with these settings.

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Fri, 22 Mar 2019 12:26:53 GMT

What's even more odd, is that If I change the .dtsi to this:

&csi1_bridge {

fsl,mipi-mode;

//fsl,two-8bit-sensor-mode;

status = "okay";

port {

csi1_ep: endpoint {

remote-endpoint = <&csi1_mipi_ep>;

data-lanes = <1 2>; <-- CHANGE RIGHT HERE (WHY!?!?)

};

&mipi_csi_1 {

#address-cells = <1>;

#size-cells = <0>;

status = "okay";

port {

mipi1_sensor_ep: endpoint1 {

remote-endpoint = <&ar0330_mipi1_ep>;

data-lanes = <1 2>;

csis-hs-settle = <1>;

csis-clk-settle = <0>;

//csis-wclk;

};

csi1_mipi_ep: endpoint2 {

remote-endpoint = <&csi1_ep>;

};

Then I get a different image:

Which leads me to believe that maybe the problem is between the MIPI-CSI2 and the CSI interface ?!

Re: iMX8M MIPI-CSI 4-lane configuration

igorpadykov — Fri, 22 Mar 2019 12:40:45 GMT

Hi Krisztian

I used sect.6.1.8 Source Code Structure attached Linux Manual.

In general recommended to rebuild all from scratch using latest nxp linux L4.14.78

from source.codeaurora.org/external/imx repository, (you are right mipi-csi2-yav.c is driver for i.MX8M).

linux-imx - i.MX Linux kernel

>fs, imx8mq-csi -> Is this something that's available somewhere

it is used in latest L4.14.78 kernel struct of_device_id mx6s_csi_dt_ids[]

mx6s_capture.c\capture\mxc\platform\media\drivers - linux-imx - i.MX Linux kernel

Regarding "csis-hs-settle" - they are not used in MX8M, they are used for MX8M Mini.

Both processors use different drivers:

"csis-hs-settle" can be found in linux/arch/arm64/boot/dts/freescale/fsl-imx8mm-evk.dts

fsl-imx8mm-evk.dts\freescale\dts\boot\arm64\arch - linux-imx - i.MX Linux kernel

linux/drivers/media/platform/mxc/capture/mxc_mipi_csi.c

mxc_mipi_csi.c\capture\mxc\platform\media\drivers - linux-imx - i.MX Linux kernel

while i.MX8M uses driver

linux/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c

mxc-mipi-csi2_yav.c\imx8\platform\media\drivers - linux-imx - i.MX Linux kernel

where HS_SETTLE parameter ((IOMUXC_GPR_GPR34) seems set with dts "phy-gpr" property in

function mxc_mipi_csi2_phy_gpr() : linux/arch/arm64/boot/dts/freescale/fsl-imx8mq.dtsi

fsl-imx8mq.dtsi\freescale\dts\boot\arm64\arch - linux-imx - i.MX Linux kernel

Best regards
igor

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Fri, 22 Mar 2019 15:41:10 GMT

Hello Julien,

Are you entirely sure that the iMX8M uses the same CSI IP as the iMX6SLL?

In the RM for imx6sll, (which is using the same IP and linux driver) the following chapter describes the IP.
"16.4.1 Data Transfer with the Embedded DMA Controllers".

I'm decoding the driver right now, and almost none of the bits match up...

Just a quick example:

CR18 register from the iMX6SLL Datasheet:

While the driver sets the data format in this register:

case V4L2_PIX_FMT_SBGGR8:

cr18 |= BIT_MIPI_DATA_FORMAT_RAW8;

Where:

#define BIT_MIPI_DATA_FORMAT_RAW8 (0x2a << 25)

Which matches up with the iMX7D datasheet for whatever reason...

It's starting to get more than confusing at this point.

Thank you in advance

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Fri, 22 Mar 2019 16:01:18 GMT

On another note, the area you referenced in the iMX8M RM, the register map references another document I cannot find:

Is this "CSI-2 Controller User Guide" available somewhere?

Re: iMX8M MIPI-CSI 4-lane configuration

krisztianbakos8 — Mon, 06 May 2019 18:40:21 GMT

I actually never got this resolved, but I feel like it might help others that I post my workaround here at-least.

So what I ended up doing is just using a single lane configuration, which takes perfect images and it's not a big deal for us right now, since we are still able to set the imager to take RAW8 or RAW12 images (both work fine after patching NXP's _yav version of the MIPI driver).

The reason for wanting multiple lanes configured was my idea if we later on would run into EMC problems, we could shift the MIPI frequencies around to avoid "dangerous" bands.. Maybe just being paranoid.

(also, I guess it should just work... but it doesn't for me, so if there is any better solution, please add it here)

Re: iMX8M MIPI-CSI 4-lane configuration

asolochek — Fri, 13 Sep 2019 15:10:50 GMT

Hi,

Can you point me to anything about the patching of the MIPI driver required to get RAW8 working? I'm about to start working on a project that will require RAW8 source support.

Re: iMX8M MIPI-CSI 4-lane configuration

evgenymolchanov — Thu, 17 Oct 2019 09:49:30 GMT

Hi Krisztian. I try get image from IMX219 sensor and haven't any interrupt at CSI-Bridge. Can you share MIPI-CSI2 and CSI_BRIDGE registers dump of your working configuration.

Re: iMX8M MIPI-CSI 4-lane configuration

paul_soucy — Wed, 23 Oct 2019 22:00:24 GMT

Hi Krisztian,

Can you share information about how you patched the MIPI driver? I am also running into the same problems. Thanks.

Re: iMX8M MIPI-CSI 4-lane configuration

felix_ye — Thu, 14 Nov 2019 09:10:36 GMT

Hi all,

Thanks to this page could I make my driver work and finally succeed to capture images of OV9716 on i.MX8MQ.

But I make it also only under the configuration of 1 data-lanes.

When data-lanes is set to 2 or 4, my image capture app would block in VIDIOC_DQBUF.

And signals of all the 4 data-lanes seem the same on the oscilloscope.

I don't know if I have expressed my problem clearly. But I would appliciate it very much if anyone could give me some advice.

Thanks

Re: iMX8M MIPI-CSI 4-lane configuration

petersonhou — Mon, 30 Dec 2019 03:22:50 GMT

what if i want to implement 4-lane configuration on IMX8QXP-MEK,any different modifications shall be taken?especially in dts,thank you!

Re: iMX8M MIPI-CSI 4-lane configuration

deepakbeniya — Thu, 27 Jun 2024 11:05:57 GMT

Can someone explain about csi1_bridge and mipi_csi_1
or give some referance video/documents/vlog that would be helpfull to understand it.

csi1_bridge: csi1_bridge@30a90000 {

compatible = "fsl,imx8mq-csi", "fsl,imx6s-csi";

reg = <0x0 0x30a90000 0x0 0x10000>;

interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>;

clocks = <&clk IMX8MQ_CLK_DUMMY>,

<&clk IMX8MQ_CLK_CSI1_ROOT>,

<&clk IMX8MQ_CLK_DUMMY>;

clock-names = "disp-axi", "csi_mclk", "disp_dcic";

dma-coherent;

status = "disabled";

};

mipi_csi_1: mipi_csi1@30a70000 {

compatible = "fsl,mxc-mipi-csi2_yav";

reg = <0x0 0x30a70000 0x0 0x1000>; /* MIPI CSI1 Controller base addr */

interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;

clocks = <&clk IMX8MQ_CLK_DUMMY>,

<&clk IMX8MQ_CLK_CSI1_CORE_DIV>,

<&clk IMX8MQ_CLK_CSI1_ESC_DIV>,

<&clk IMX8MQ_CLK_CSI1_PHY_REF_DIV>;

clock-names = "clk_apb", "clk_core", "clk_esc", "clk_pxl";

assigned-clocks = <&clk IMX8MQ_CLK_CSI1_CORE_DIV>,

<&clk IMX8MQ_CLK_CSI1_PHY_REF_DIV>,

<&clk IMX8MQ_CLK_CSI1_ESC_DIV>;

assigned-clock-rates = <133000000>, <100000000>, <66000000>;

power-domains = <&mipi_csi1_pd>;

csis-phy-reset = <&src 0x4c 7>;

phy-gpr = <&gpr 0x88>;

status = "disabled";

};