On i.MX8QM MIPI CSI-2, no interrupt at ISI.

JohnLinuxer92 · ‎11-28-2023

Hello. We made board with referring i.MX8QMMEK.

After we modified device-tree, here's the result.

root@imx8qmmek:~# dmesg | grep "mxc"
[ 1.976687] mxc-isi 58100000.isi: mxc_isi.0 registered successfully
[ 1.983583] mxc-isi 58110000.isi: mxc_isi.1 registered successfully
[ 1.997486] mxc-isi 58120000.isi: mxc_isi.2 registered successfully
[ 2.004343] mxc-isi 58130000.isi: mxc_isi.3 registered successfully
[ 2.011198] mxc-isi 58140000.isi: mxc_isi.4 registered successfully
[ 2.018046] mxc-isi 58150000.isi: mxc_isi.5 registered successfully
[ 2.024911] mxc-isi 58160000.isi: mxc_isi.6 registered successfully
[ 2.031747] mxc-isi 58170000.isi: mxc_isi.7 registered successfully
[ 2.564748] mxc-mipi-csi2 58227000.csi: lanes: 4, name: mxc-mipi-csi2.0
[ 2.592724] mxc-mipi-csi2 58247000.csi: lanes: 4, name: mxc-mipi-csi2.1
[ 8.922146] mx8-img-md: Registered mxc_isi.0.capture as /dev/video2
[ 8.939080] mx8-img-md: Registered mxc_isi.1.capture as /dev/video3
[ 8.955665] mx8-img-md: Registered mxc_isi.2.capture as /dev/video4
[ 8.967181] mx8-img-md: Registered mxc_isi.3.capture as /dev/video5
[ 9.013098] mx8-img-md: Registered mxc_isi.4.capture as /dev/video6
[ 9.021249] mx8-img-md: Registered mxc_isi.5.capture as /dev/video7
[ 9.030102] mx8-img-md: Registered mxc_isi.6.capture as /dev/video8
[ 9.042226] mx8-img-md: Registered mxc_isi.7.capture as /dev/video9
[ 9.071250] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 9.079905] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 9.088083] mx8-img-md: created link [mxc_isi.1] => [mxc_isi.1.capture]
[ 9.096928] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.1]
[ 9.105247] mx8-img-md: created link [mxc_isi.2] => [mxc_isi.2.capture]
[ 9.115808] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.2]
[ 9.127793] mx8-img-md: created link [mxc_isi.3] => [mxc_isi.3.capture]
[ 9.142346] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.3]
[ 9.153588] mx8-img-md: created link [mxc_isi.4] => [mxc_isi.4.capture]
[ 9.167270] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.4]
[ 9.181246] mx8-img-md: created link [mxc_isi.5] => [mxc_isi.5.capture]
[ 9.234037] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.5]
[ 9.240650] mx8-img-md: created link [mxc_isi.6] => [mxc_isi.6.capture]
[ 9.252993] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.6]
[ 9.260534] mx8-img-md: created link [mxc_isi.7] => [mxc_isi.7.capture]
[ 9.275835] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.7]
[ 9.275850] mx8-img-md: created link [eocam_mipi 2-006a] => [mxc-mipi-csi2.0]
[ 9.299886] mx8-img-md: created link [ircam_mipi 3-003c] => [mxc-mipi-csi2.1]
[ 9.317470] mxc-md bus@58000000:camera: mxc_md_create_links
[ 9.469091] mxc-jpeg 58400000.jpegdec: decoder device registered as /dev/video10 (81,14)
[ 9.496812] mxc-jpeg 58450000.jpegenc: encoder device registered as /dev/video11 (81,15)

eocam and ircam are our v4l2 device driver to receive MIPI CSI frame.
It seems there's no error at all. But when we check to see camera frame using Gstreamer, there's no frame.

So, we set printk message on imx8-isi-core.c's interrupt handler named 'mxc_isi_irq_handler'.

But there's no printk message displayed on kernel console. It means, there's no interrupt occurred.

I don't know why.

The below is our MIPI device-tree.

&mipi_csi_0 {

#address-cells = <1>;

#size-cells = <0>;

// virtual-channel;

status = "okay";

/* Camera 0 MIPI CSI-2 (CSIS0) */

port@0 {

reg = <0>;

mipi_csi0_ep: endpoint {

remote-endpoint = <&eocam_mipi_ep>;

data-lanes = <1 2 3 4>;

bus-type = <4>;

};

&i2c_mipi_csi0 {

#address-cells = <1>;

#size-cells = <0>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_i2c_mipi_csi0>;

clock-frequency = <100000>;

status = "okay";

eocam_mipi: eocam_mipi@6a {

compatible = "eocam_mipi";

reg = <0x6a>;

mipi_csi;

csi_id = <0>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_mipi_csi0>;

clocks = <&clk_dummy>;

clock-names = "capture_mclk";

mclk = <27000000>;

mclk_source = <0>;

pwn-gpios = <&lsio_gpio1 27 GPIO_ACTIVE_HIGH>;

status = "okay";

port {

eocam_mipi_ep: endpoint {

remote-endpoint = <&mipi_csi0_ep>;

data-lanes = <1 2 3 4>;

clock-lanes = <0>;

};

&isi_0 {

status = "okay";

cap_device {

status = "okay";

};

m2m_device {

// status = "okay";

status = "disabled";

};

&isi_1 {

status = "okay";

cap_device {

status = "okay";

};

&isi_2 {

status = "okay";

cap_device {

status = "okay";

};

&isi_3 {

status = "okay";

cap_device {

status = "okay";

};

I don't know why there's no interrupt at ISI module.

JohnLinuxer92 · ‎12-21-2023

Dear community,

I figured out why there's no interrupt at ISI.

It was because of poor signal integrity + differential impedance matching of our board's connector/PCB pattern.

After temporary HW handling, I finally got intermittent interrupt.

Our HW team now working on this to solve the issue totally.

There was no reason that MIPI doesn't work in terms of SW. It's been a exciting and rough journey.

Thanks!

View solution in original post

joanxie · ‎11-29-2023

do you mind sharing the result when you use "v4l2-ctl --list-device"

JohnLinuxer92 · ‎11-29-2023

Hello joanxie, here's the v4l2-ctl --list-device result that you mentioned.

amphion vpu decoder (platform: amphion-vpu):
/dev/video0
/dev/video1
/dev/media0

mxc-isi-cap (platform:58100000.isi:cap_devic):
/dev/video2

mxc-isi-cap (platform:58110000.isi:cap_devic):
/dev/video3

mxc-isi-cap (platform:58120000.isi:cap_devic):
/dev/video4

mxc-isi-cap (platform:58130000.isi:cap_devic):
/dev/video5

mxc-isi-cap (platform:58140000.isi:cap_devic):
/dev/video6

mxc-isi-cap (platform:58150000.isi:cap_devic):
/dev/video7

mxc-isi-cap (platform:58160000.isi:cap_devic):
/dev/video8

mxc-isi-cap (platform:58170000.isi:cap_devic):
/dev/video9

mxc-jpeg codec (platform:58400000.jpegdec):
/dev/video10

mxc-jpeg codec (platform:58450000.jpegenc):
/dev/video11

FSL Capture Media Device (platform:mxc-md):
/dev/media1

Thanks.

JohnLinuxer92 · ‎11-29-2023

And Here's the gst-launch-1.0 v4l2src device=/dev/video2 ! autovideoconvert ! autovideosink result.

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...[ 26.671442] isi-capture 58100000.isi:cap_devic

Pipeline is PREROLLED ...[ 26.681831] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_f

Setting pipeline to PLAYING ...
New clock: GstSystemClock[ 26.690599] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_f

[ 26.702392] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.708909] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.715382] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.721850] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.728328] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.734800] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.741273] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.747754] isi-capture 58100000.isi:cap_device: mxc_isi_cap_enum_fmt
[ 26.754231] isi-capture 58100000.isi:cap_device: mxc_isi_cap_g_selection
No g2d kernel cache file /tmp/g2d_opencl_kernel.bin
Building g2d kernel ... buildBinarySize = 59327
[ 27.034109] isi-capture 58100000.isi:cap_device: mxc_isi_cap_try_fmt_mplane
[ 27.051845] isi-capture 58100000.isi:cap_device: mxc_isi_cap_s_fmt_mplane, fmt=0x56595559
[ 27.060082] isi-capture 58100000.isi:cap_device: mxc_isi_cap_try_fmt_mplane
[ 27.067584] isi-capture 58100000.isi:cap_device: mxc_isi_cap_s_fmt_mplane, fmt=0x56595559
[ 27.075793] isi-capture 58100000.isi:cap_device: mxc_isi_cap_try_fmt_mplane
[ 27.082970] isi-capture 58100000.isi:cap_device: cap_vb2_queue_setup, buf_n=3, size=153600
[ 27.092696] isi-capture 58100000.isi:cap_device: cap_vb2_buffer_prepare
[ 27.099378] isi-capture 58100000.isi:cap_device: cap_vb2_buffer_prepare
[ 27.106058] isi-capture 58100000.isi:cap_device: cap_vb2_buffer_prepare
[ 27.112737] isi-capture 58100000.isi:cap_device: mxc_isi_cap_streamon
[ 27.144980] mxc-mipi-csi2 58227000.csi: width=320, height=240, fmt.code=0x2008
[ 27.157233] bypass csc
[ 27.159592] input fmt YUV4
[ 27.162322] output fmt YUYV
[ 27.165140] mxc-isi 58100000.isi: mxc_isi_channel_set_scaling: no scale
[ 27.171782] isi-capture 58100000.isi:cap_device: cap_vb2_start_streaming
[ 27.178933] isi-capture 58100000.isi:cap_device: cap_vb2_start_streaming: num_plane=0 discar
[ 27.524213] isi-capture 58100000.isi:cap_device: mxc_isi.1.capture is no v4l2 subdev
[ 27.531979] isi-capture 58100000.isi:cap_device: mxc_isi.2.capture is no v4l2 subdev
[ 27.539752] isi-capture 58100000.isi:cap_device: mxc_isi.3.capture is no v4l2 subdev
[ 27.547524] eocam_mipi 2-006a: s_stream: 1
[ 27.557055] mxc-mipi-csi2 58227000.csi: mipi_csi2_s_stream: 1, csi2dev: 0x0
[ 27.577212] mxc-mipi-csi2 58227000.csi: width=320, height=240, fmt.code=0x2008
[ 27.584464] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR and HC register dump, mipi csi0
[ 27.592317] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC num of lanes[0x100]: 0x003
[ 27.599649] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC dis lanes[0x104]: 0x000
[ 27.606720] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC BIT ERR[0x108]: 0x000
[ 27.613619] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC IRQ STATUS[0x10c]: 0x008
[ 27.620777] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC IRQ MASK[0x110]: 0x1ff
[ 27.627761] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC ULPS STATUS[0x114]: 0x000
[ 27.635007] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DPHY ErrSotHS[0x118]: 0x000
[ 27.642425] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DPHY ErrSotSync[0x11c]: 0x000
[ 27.650019] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DPHY ErrEsc[0x120]: 0x000
[ 27.657263] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DPHY ErrSyncEsc[0x124]: 0x000
[ 27.664857] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DPHY ErrControl[0x128]: 0x000
[ 27.672451] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DISABLE_PAYLOAD[0x12c]: 0x000
[ 27.680047] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC DISABLE_PAYLOAD[0x130]: 0x000
[ 27.687637] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC IGNORE_VC[0x180]: 0x000
[ 27.694714] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC VID_VC[0x184]: 0x000
[ 27.701614] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC FIFO_SEND_LEVEL[0x188]: 0x000
[ 27.709207] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC VID_VSYNC[0x18c]: 0x000
[ 27.716278] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC VID_SYNC_FP[0x190]: 0x000
[ 27.723523] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC VID_HSYNC[0x194]: 0x000
[ 27.730595] mxc-mipi-csi2 58227000.csi: MIPI CSI2 HC VID_HSYNC_BP[0x198]: 0x000
[ 27.737928] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PLM_CTRL[0x000]: 0x000
[ 27.744999] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_CTRL[0x004]: 0x000
[ 27.752071] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_Status[0x008]: 0x000
[ 27.759319] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_Test_Status[0x010]: 0x000
[ 27.766996] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_Test_Status[0x014]: 0x000
[ 27.774678] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_Test_Status[0x018]: 0x000
[ 27.782356] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_Test_Status[0x01c]: 0x000
[ 27.790039] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR PHY_Test_Status[0x020]: 0x000
[ 27.797717] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR VC Interlaced[0x030]: 0x000
[ 27.805221] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR Data Type Dis[0x038]: 0x000
[ 27.812727] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR 420 1st type[0x040]: 0x000
[ 27.820155] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR Ctr_Ck_Rst_Ctr[0x044]: 0x000
[ 27.827748] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR Stream Fencing[0x048]: 0x000
[ 27.835341] mxc-mipi-csi2 58227000.csi: MIPI CSI2 CSR Stream Fencing[0x04c]: 0x000
[ 27.842934] isi-capture 58100000.isi:cap_device: mxc_isi.0.capture is no v4l2 subdev

I used dynamic debug to built-in kernel source and it seems there's no register set on MIPI CSI2 CSR.

Our input is 1920x1080 for valid(total 2200x1125) with 60fps to MIPI_CSI_0. But according to ERR050066, I understood that this bandwidth can cause problem. Should less than 2Mpixel, right?

Because of this reason, is it possible that interrupt doesn't occur?

joanxie · ‎12-04-2023

we only tested 4x1280*800@30Hz, how about just enable one camera? is it ok? default bsp uses 24M mclk, what clock do you set for your new mclk now?

JohnLinuxer92 · ‎12-04-2023

Actaully, that camera is FPGA. FPGA sends video frame. And mclk is unnecessary for our topology.

If 1280*800@30Hz with 4lane is only tested, 1920x1080@60Hz with 4lane can have a issue like this interrupt things?

joanxie · ‎12-04-2023

not the same issue, refer to your settings, you use virtual channel, I need more your project information, I mailed to you pls check

JohnLinuxer92 · ‎12-04-2023

Hello joan, I replied with my mail.
Thanks for your effort

JohnLinuxer92 · ‎11-29-2023

In the other post, the MIPI CSI2 CSR PLM_CTRL[0x000] is 0x801, but why My MIPI CSI2 CSR PLM_CTRL[0x000] is 0x000?
In the code below, code set registers including PLM_CTRL.

static void mxc_mipi_csi2_enable(struct mxc_mipi_csi2_dev *csi2dev)

{

struct device *dev = &csi2dev->pdev->dev;

u32 val = 0;

val = readl(csi2dev->csr_regs + CSI2SS_PLM_CTRL);

while (val & CSI2SS_PLM_CTRL_PL_CLK_RUN) {

msleep(10);

val = readl(csi2dev->csr_regs + CSI2SS_PLM_CTRL);

dev_dbg(dev, "Waiting pl clk running, val=0x%x\n", val);

}

/* Enable Pixel link Master*/

val = readl(csi2dev->csr_regs + CSI2SS_PLM_CTRL);

val |= CSI2SS_PLM_CTRL_ENABLE_PL;

writel(val, csi2dev->csr_regs + CSI2SS_PLM_CTRL);

val |= CSI2SS_PLM_CTRL_VALID_OVERRIDE;

writel(val, csi2dev->csr_regs + CSI2SS_PLM_CTRL);

/* PHY Enable */

val = readl(csi2dev->csr_regs + CSI2SS_PHY_CTRL);

val &= ~(CSI2SS_PHY_CTRL_PD_MASK | CSI2SS_PLM_CTRL_POLARITY_MASK);

writel(val, csi2dev->csr_regs + CSI2SS_PHY_CTRL);

/* Deassert reset */

writel(1, csi2dev->csr_regs + CSI2SS_CTRL_CLK_RESET);

}

joanxie · ‎12-12-2023

thanks for your dts files, I reviewed them and I found you disable the virtual channels, since you use all of ISI, you need enable the virtual channels to map the video from mipi csi to the different ISI, pls enable it and try it again

JohnLinuxer92 · ‎12-12-2023

Actually, Our camera#1 which has MIPI CSI-2 4-lane will connect to MIPI CSI-2 ch0 on i.MX8QM, camera#2 which also has same 4-lane will connect to MIPI CSI-2 ch1.

So, I disabled virtual-channel property in device-tree and also, I originally set all isi0-isi7 property but as you hinted about virtual-channel and isi things, I disabled isi1 to isi3 and isi5 to isi7, only set isi0(for MIPI CSI-2 ch0) and isi4(for MIPI CSI-2 ch1). Like the below.

&isi_0 {

status = "okay";

cap_device {

status = "okay";

};

m2m_device {

// status = "okay";

status = "disabled";

};

&isi_1 {

// status = "okay";

status = "disabled";

cap_device {

// status = "okay";

status = "disabled";

};

&isi_2 {

// status = "okay";

status = "disabled";

cap_device {

// status = "okay";

status = "disabled";

};

&isi_3 {

// status = "okay";

status = "disabled";

cap_device {

// status = "okay";

status = "disabled";

};

&isi_4 {

status = "okay";

cap_device {

status = "okay";

};

&isi_5 {

// status = "okay";

status = "disabled";

cap_device {

// status = "okay";

status = "disabled";

};

&isi_6 {

// status = "okay";

status = "disabled";

cap_device {

// status = "okay";

status = "disabled";

};

&isi_7 {

// status = "okay";

status = "disabled";

cap_device {

// status = "okay";

status = "disabled";

};

But, above aspect doesn't work. Still isi interrupt doesn't occurred.

joanxie · ‎12-14-2023

I suggest that you can refer to the nxp dts file as below which enable two camera to the different MIPI CSI

https://github.com/nxp-imx/linux-imx/blob/lf-6.1.y/arch/arm64/boot/dts/freescale/imx8qm-mek-ov5640.d...

https://github.com/nxp-imx/linux-imx/blob/lf-6.1.y/arch/arm64/boot/dts/freescale/imx8qm-mek.dts

JohnLinuxer92 · ‎12-14-2023

If input MIPI CSI-2 video frame's waveform is distorted or level is not matched with MIPI standard, does the below scenario make sense??

1. Pixel DMA in isi will not recognize input as video frame

2. For isi, there's no input video frame that recognized, DMA will not work

3. So, pixel DMA will not generate interrupt to GiC, so isi interrupt handler will never invoke.

Thanks.

JohnLinuxer92 · ‎12-19-2023

In our design, we use FPGA as camera that start sending frame without configure using i2c.

So, we removed i2c.

In our device-tree, we put our camera property at "i2c_mipi_csi0" which there's nothing connected to that i2c pin, just floating in hardware.

Is this matter?

I don't know why isi's interrupt doesn't occurred when input MIPI frame waveform quality is fine.

JohnLinuxer92 · ‎12-21-2023

Dear community,

I figured out why there's no interrupt at ISI.

It was because of poor signal integrity + differential impedance matching of our board's connector/PCB pattern.

After temporary HW handling, I finally got intermittent interrupt.

Our HW team now working on this to solve the issue totally.

There was no reason that MIPI doesn't work in terms of SW. It's been a exciting and rough journey.

Thanks!