Camera with specific resolution by ISI couldn't preview on screen.

[jerry_liu]

Hello,

We use onsemi sensor and ISP (AR0821+AP1302) on imx8M plus.

we can enable the camera with CSI0 to support 4K, 2K, 1080p, and 720p.

However, when we tried the same camera with CSI1, it had a problem so that it couldn't support 720p.

Because the camera output resolution is decided by AP1302, so we add more resolutions to try, and we found some specific resolutions cannot be supported.

Below we tried:

Resolution	CSI0 (/dev/video0)	CSI1 (/dev/video1)
3840x2160	V	V
2560x1440	V	V
1920x1200	V	V
1920x1080	V	V
1600x1200	V	X
1280x800	X	X
1280x720	V	X
1024x768	X	X
800x600	V	X
720x480	V	X
640x400	X	X
640x360	V	X

We wonder if it was the ISI problem but we couldn't find any problem even we printed logs of ISI.

we only knew the mxc_isi_irq_handler didn't work if we set the specific resolutions.

CSI0: (/dev/video0 1280x720 can work)

[  127.754846] mxc-isi 32e00000.isi: ISI CHNLC register dump, isi0
[  127.760829] mxc-isi 32e00000.isi:            CHNL_CTRL[0x00]: e0000000
[  127.767380] mxc-isi 32e00000.isi:        CHNL_IMG_CTRL[0x04]: 20000001
[  127.773925] mxc-isi 32e00000.isi:    CHNL_OUT_BUF_CTRL[0x08]: 7c707
[  127.780205] mxc-isi 32e00000.isi:         CHNL_IMG_CFG[0x0c]: 4380780
[  127.786662] mxc-isi 32e00000.isi:             CHNL_IER[0x10]: 3cfc0000
[  127.793204] mxc-isi 32e00000.isi:             CHNL_STS[0x14]: 100
[  127.799324] mxc-isi 32e00000.isi:    CHNL_SCALE_FACTOR[0x18]: 10001000
[  127.805876] mxc-isi 32e00000.isi:    CHNL_SCALE_OFFSET[0x1c]: 00
[  127.811895] mxc-isi 32e00000.isi:        CHNL_CROP_ULC[0x20]: 00
[  127.817917] mxc-isi 32e00000.isi:        CHNL_CROP_LRC[0x24]: 00
[  127.823940] mxc-isi 32e00000.isi:      CHNL_CSC_COEFF0[0x28]: 00
[  127.829979] mxc-isi 32e00000.isi:      CHNL_CSC_COEFF1[0x2c]: 00
[  127.836009] mxc-isi 32e00000.isi:      CHNL_CSC_COEFF2[0x30]: 00
[  127.842048] mxc-isi 32e00000.isi:      CHNL_CSC_COEFF3[0x34]: 00
[  127.848076] mxc-isi 32e00000.isi:      CHNL_CSC_COEFF4[0x38]: 00
[  127.854100] mxc-isi 32e00000.isi:      CHNL_CSC_COEFF5[0x3c]: 00
[  127.860122] mxc-isi 32e00000.isi:     CHNL_ROI_0_ALPHA[0x40]: 00
[  127.866144] mxc-isi 32e00000.isi:       CHNL_ROI_0_ULC[0x44]: 00
[  127.872166] mxc-isi 32e00000.isi:       CHNL_ROI_0_LRC[0x48]: 00
[  127.878187] mxc-isi 32e00000.isi:     CHNL_ROI_1_ALPHA[0x4c]: 00
[  127.884219] mxc-isi 32e00000.isi:       CHNL_ROI_1_ULC[0x50]: 00
[  127.890241] mxc-isi 32e00000.isi:       CHNL_ROI_1_LRC[0x54]: 00
[  127.896263] mxc-isi 32e00000.isi:     CHNL_ROI_2_ALPHA[0x58]: 00
[  127.902293] mxc-isi 32e00000.isi:       CHNL_ROI_2_ULC[0x5c]: 00
[  127.908319] mxc-isi 32e00000.isi:       CHNL_ROI_2_LRC[0x60]: 00
[  127.914341] mxc-isi 32e00000.isi:     CHNL_ROI_3_ALPHA[0x64]: 00
[  127.920365] mxc-isi 32e00000.isi:       CHNL_ROI_3_ULC[0x68]: 00
[  127.926386] mxc-isi 32e00000.isi:       CHNL_ROI_3_LRC[0x6c]: 00
[  127.932408] mxc-isi 32e00000.isi: CHNL_OUT_BUF1_ADDR_Y[0x70]: 68900000
[  127.938950] mxc-isi 32e00000.isi: CHNL_OUT_BUF1_ADDR_U[0x74]: 00
[  127.944972] mxc-isi 32e00000.isi: CHNL_OUT_BUF1_ADDR_V[0x78]: 00
[  127.950994] mxc-isi 32e00000.isi:   CHNL_OUT_BUF_PITCH[0x7c]: f00
[  127.957103] mxc-isi 32e00000.isi:     CHNL_IN_BUF_ADDR[0x80]: 00
[  127.963123] mxc-isi 32e00000.isi:    CHNL_IN_BUF_PITCH[0x84]: 00
[  127.969145] mxc-isi 32e00000.isi:     CHNL_MEM_RD_CTRL[0x88]: 00
[  127.975167] mxc-isi 32e00000.isi: CHNL_OUT_BUF2_ADDR_Y[0x8c]: 67900000
[  127.981708] mxc-isi 32e00000.isi: CHNL_OUT_BUF2_ADDR_U[0x90]: 00
[  127.987725] mxc-isi 32e00000.isi: CHNL_OUT_BUF2_ADDR_V[0x94]: 00
[  127.993748] mxc-isi 32e00000.isi:     CHNL_SCL_IMG_CFG[0x98]: 4380780
[  128.000204] mxc-isi 32e00000.isi:       CHNL_FLOW_CTRL[0x9c]: 00
[  128.477737] mxc-isi 32e00000.isi: mxc_isi_irq_handler
[  128.512879] mxc-isi 32e00000.isi: mxc_isi_irq_handler

CSI1: (/dev/video1 1280x720 cannot work)

[   28.305635] mxc-isi 32e02000.isi: ISI CHNLC register dump, isi1
[   28.311589] mxc-isi 32e02000.isi:            CHNL_CTRL[0x00]: e0000001
[   28.318130] mxc-isi 32e02000.isi:        CHNL_IMG_CTRL[0x04]: 20000001
[   28.324673] mxc-isi 32e02000.isi:    CHNL_OUT_BUF_CTRL[0x08]: 7c707
[   28.330953] mxc-isi 32e02000.isi:         CHNL_IMG_CFG[0x0c]: 2d00500
[   28.337439] mxc-isi 32e02000.isi:             CHNL_IER[0x10]: 3cfc0000
[   28.343987] mxc-isi 32e02000.isi:             CHNL_STS[0x14]: 100
[   28.350104] mxc-isi 32e02000.isi:    CHNL_SCALE_FACTOR[0x18]: 10001000
[   28.356651] mxc-isi 32e02000.isi:    CHNL_SCALE_OFFSET[0x1c]: 00
[   28.362687] mxc-isi 32e02000.isi:        CHNL_CROP_ULC[0x20]: 00
[   28.368708] mxc-isi 32e02000.isi:        CHNL_CROP_LRC[0x24]: 00
[   28.374746] mxc-isi 32e02000.isi:      CHNL_CSC_COEFF0[0x28]: 00
[   28.380770] mxc-isi 32e02000.isi:      CHNL_CSC_COEFF1[0x2c]: 00
[   28.386804] mxc-isi 32e02000.isi:      CHNL_CSC_COEFF2[0x30]: 00
[   28.392839] mxc-isi 32e02000.isi:      CHNL_CSC_COEFF3[0x34]: 00
[   28.398875] mxc-isi 32e02000.isi:      CHNL_CSC_COEFF4[0x38]: 00
[   28.404908] mxc-isi 32e02000.isi:      CHNL_CSC_COEFF5[0x3c]: 00
[   28.410946] mxc-isi 32e02000.isi:     CHNL_ROI_0_ALPHA[0x40]: 00
[   28.416984] mxc-isi 32e02000.isi:       CHNL_ROI_0_ULC[0x44]: 00
[   28.423021] mxc-isi 32e02000.isi:       CHNL_ROI_0_LRC[0x48]: 00
[   28.429054] mxc-isi 32e02000.isi:     CHNL_ROI_1_ALPHA[0x4c]: 00
[   28.435109] mxc-isi 32e02000.isi:       CHNL_ROI_1_ULC[0x50]: 00
[   28.441146] mxc-isi 32e02000.isi:       CHNL_ROI_1_LRC[0x54]: 00
[   28.447184] mxc-isi 32e02000.isi:     CHNL_ROI_2_ALPHA[0x58]: 00
[   28.453210] mxc-isi 32e02000.isi:       CHNL_ROI_2_ULC[0x5c]: 00
[   28.459246] mxc-isi 32e02000.isi:       CHNL_ROI_2_LRC[0x60]: 00
[   28.465272] mxc-isi 32e02000.isi:     CHNL_ROI_3_ALPHA[0x64]: 00
[   28.471307] mxc-isi 32e02000.isi:       CHNL_ROI_3_ULC[0x68]: 00
[   28.477333] mxc-isi 32e02000.isi:       CHNL_ROI_3_LRC[0x6c]: 00
[   28.483368] mxc-isi 32e02000.isi: CHNL_OUT_BUF1_ADDR_Y[0x70]: 68100000
[   28.489914] mxc-isi 32e02000.isi: CHNL_OUT_BUF1_ADDR_U[0x74]: 00
[   28.495945] mxc-isi 32e02000.isi: CHNL_OUT_BUF1_ADDR_V[0x78]: 00
[   28.501966] mxc-isi 32e02000.isi:   CHNL_OUT_BUF_PITCH[0x7c]: a00
[   28.508075] mxc-isi 32e02000.isi:     CHNL_IN_BUF_ADDR[0x80]: 00
[   28.514095] mxc-isi 32e02000.isi:    CHNL_IN_BUF_PITCH[0x84]: 00
[   28.520127] mxc-isi 32e02000.isi:     CHNL_MEM_RD_CTRL[0x88]: 00
[   28.526153] mxc-isi 32e02000.isi: CHNL_OUT_BUF2_ADDR_Y[0x8c]: 67900000
[   28.532696] mxc-isi 32e02000.isi: CHNL_OUT_BUF2_ADDR_U[0x90]: 00
[   28.538716] mxc-isi 32e02000.isi: CHNL_OUT_BUF2_ADDR_V[0x94]: 00
[   28.544748] mxc-isi 32e02000.isi:     CHNL_SCL_IMG_CFG[0x98]: 2d00500
[   28.551208] mxc-isi 32e02000.isi:       CHNL_FLOW_CTRL[0x9c]: 00

To confirm what happens, we tried it on imx8M Mini (without ISI).

We found the above resolutions all support on imx8M Mini except 2K and 4K.

So we think the problem is from ISI.

How can we resolve the problem?

Thanks!

[joanxie]

I tested ov5640 on mipi csi2 port ( mxc-isi 32e02000.isi), tested 720x480 and 1280x720 successfully, but 1280x800 failed, because camera doesn't support this resolution, so I don't think ISI has this limitation,  I tested these resolution successfully

[3]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 176x144
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1024x768
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)

[jerry_liu]

Ok, I tried to get the support format from /dev/video0 and /dev/video1.

I found there are some lost resolutions on video1. I think it should be the same.

But I don't know why they are different.

video0:

root@edm-g-imx8mp:~# v4l2-ctl --device /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'RGBP' (16-bit RGB 5-6-5)
        [1]: 'RGB3' (24-bit RGB 8-8-8)
        [2]: 'BGR3' (24-bit BGR 8-8-8)
        [3]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x400
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 720x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 800x600
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1024x768
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x800
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1600x1200
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1200
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2560x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2600x1952
                        Interval: Discrete 0.033s (30.000 fps)
        [4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
        [5]: 'NV12' (Y/CbCr 4:2:0)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x400
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 720x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 800x600
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1024x768
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x800
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1600x1200
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1200
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2560x1440
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2600x1952
                        Interval: Discrete 0.033s (30.000 fps)
        [6]: 'YM24' (Planar YUV 4:4:4 (N-C))
        [7]: 'XR24' (32-bit BGRX 8-8-8-8)
        [8]: 'AR24' (32-bit BGRA 8-8-8-8)

video1:

root@edm-g-imx8mp:~# v4l2-ctl --device /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'RGBP' (16-bit RGB 5-6-5)
        [1]: 'RGB3' (24-bit RGB 8-8-8)
        [2]: 'BGR3' (24-bit BGR 8-8-8)
        [3]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x400
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 720x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 800x600
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1024x768
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x800
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1600x1200
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1200
                        Interval: Discrete 0.033s (30.000 fps)
        [4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
        [5]: 'NV12' (Y/CbCr 4:2:0)
                Size: Discrete 640x360
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x400
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 720x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 800x600
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1024x768
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x800
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1600x1200
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1200
                        Interval: Discrete 0.033s (30.000 fps)
        [6]: 'YM24' (Planar YUV 4:4:4 (N-C))
        [7]: 'XR24' (32-bit BGRX 8-8-8-8)
        [8]: 'AR24' (32-bit BGRA 8-8-8-8)

[joanxie]

for your own camera, what source code do you change? like dtb or kernel? isi0 and isi1 are different setting in the default dtb, if you change anything, could you shar with me?

[jerry_liu]

Yes, sure.

But the dtb is fixed to resolve a bug about the wrong image when the camera is previewed above 2K resolution on CSI1(/dev/video1).

From 3ea3f8e72f966ac974aeb5c243bb27ae272e8aca Mon Sep 17 00:00:00 2001
From: JerryLiu <jerry.liu@technexion.com>
Date: Wed, 15 Dec 2021 09:39:25 +0800
Subject: [PATCH] ARM64: dts: imx8mp: modified isi_1 config to fixed image
 distortion problem.

---
 arch/arm64/boot/dts/freescale/imx8mp-edm-g.dtsi |  4 ++++
 arch/arm64/boot/dts/freescale/imx8mp.dtsi       | 15 +++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/boot/dts/freescale/imx8mp-edm-g.dtsi b/arch/arm64/boot/dts/freescale/imx8mp-edm-g.dtsi
index 4443dec4376f..60069de4ed3f 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp-edm-g.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp-edm-g.dtsi
@@ -996,6 +996,10 @@
 	cap_device {
 		status = "okay";
 	};
+
+	m2m_device {
+		status = "okay";
+	};
 };
 
 &dsp {
diff --git a/arch/arm64/boot/dts/freescale/imx8mp.dtsi b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
index e3ec280b53d0..30e2ed28c3b2 100644
--- a/arch/arm64/boot/dts/freescale/imx8mp.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mp.dtsi
@@ -1742,7 +1742,12 @@
 				reg = <0x32ec0008 0x4>;
 			};
 
-			isi_chain_buf: isi_chain@32e02000{
+			isi_chain_buf0: isi_chain0@32e00000{
+				compatible = "fsl,imx8mp-iomuxc-gpr", "syscon";
+				reg = <0x32e00000 0x4>;
+			};
+
+			isi_chain_buf1: isi_chain1@32e02000{
 				compatible = "fsl,imx8mp-iomuxc-gpr", "syscon";
 				reg = <0x32e02000 0x4>;
 			};
@@ -1769,7 +1774,7 @@
 					assigned-clock-rates = <500000000>, <200000000>;
 					no-reset-control;
 					power-domains = <&mediamix_pd>;
-					isi_chain = <&isi_chain_buf>;
+					isi_chain = <&isi_chain_buf0>;
 					status = "disabled";
 
 					cap_device {
@@ -1798,12 +1803,18 @@
 					assigned-clock-rates = <500000000>, <200000000>;
 					no-reset-control;
 					power-domains = <&mediamix_pd>;
+					isi_chain = <&isi_chain_buf1>;
 					status = "disabled";
 
 					cap_device {
 						compatible = "imx-isi-capture";
 						status = "disabled";
 					};
+
+					m2m_device{
+						compatible = "imx-isi-m2m";
+						status = "disabled";
+					};
 				};
 
 				isp_0: isp@32e10000 {
-- 
2.25.1

[joanxie]

the latest bsp should fix this issue

[jerry_liu · Accepted solution]

Hello,

I found this issue was not related to ISI. The cause is just the clock frequencies are different between mipi_csi0 and  mipi_csi_1 so that it makes some resolutions cannot preview.

[joanxie]

yes,mipi csi1 and mipi csi0 doesn't have the same clock, but for mipi csi1 has 266M, how could your mipi csi1 couldn't capture even 640x480 resolution successfully?

[jerry_liu]

I don't know why it couldn't preview some resolutions with 266M but it surely happened.

Could you tell me how to set the clock frequency of mipi_csi?

Is it related to the camera's PLL clock setting?

[joanxie]

 clock is set in the dts file, 2k should related to clock, but you couldn't support 720x480 and 1280x720 , I tested them successfully, I don't think this issue related to the clock