camera on imx93 board

[zhangjunsheng]

Hi,

     I am try to turn on topaz2m camera on imx93 platform board. when I  try to drive camera to output data, v412-ctl is stuck. I enabled the debug printing in CSI driver, get below result. 

# v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=GREY; setsid -f v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=1

topaz2m 1-0010: mbus_fmt: 2001
mxc-mipi-csi2.0: format: 0x2001
bypass csc
input fmt GREY
output fmt GREY
dwc-mipi-csi2-host 4ae00000.csi: enter enable=1
dwc-mipi-csi2-host 4ae00000.csi: data_type:0x1e, virtual chan: 0
dwc-mipi-csi2-host 4ae00000.csi: hsa_time=0, hbp_time=0, hsd_time=0, hline_time=1280
dwc-mipi-csi2-host 4ae00000.csi: ipi mode: camera, color_mode: color mode 48
dwc-mipi-csi2-host 4ae00000.csi: format: 0x2001, w/h=(1920, 1080)
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 Version: 0x3135302a
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 lanes: 0x1
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 HOST RESETN: 0x1
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT STATUS MAIN: 0x40000
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DATA IDS1: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DATA IDS2: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY CFG: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY MODE: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT STATUS AP MAIN: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DATA IDS VC1: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DATA IDS VC2: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY SHUTDOWN: 0x1
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY RESET: 0x1
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY RX STATUS: 0x30000
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY STOP STATUS: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY TEST CTRL0: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 DPHY TEST CTRL1: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 PPI PG PATTERN HIGH: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 PPI PG PATTERN WIDTH: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 PPI PG CONFIG: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 PPI PG ENABLE: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 PPI PG STATUS: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 IPI MODE: 0x1010000
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 IPI VCID: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 IPI DATA TYPE: 0x1e
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 IPI SOFT RESET: 0x1
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 IPI ADV FEATURE: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_PHY_FATAL: 0x3
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_PKT_FATAL: 0x3
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_PHY: 0x3
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_IPI_FATAL: 0x7f
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_BNDRY_FRAME_FATAL: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_SEQ_FRAME_FATAL: 0x0
dwc-mipi-csi2-host 4ae00000.csi: DWC CSI2 INT_MSK_DATA_ID: 0x0
dwc-mipi-csi2-host 4ae00000.csi: gasket: CAMERA MUX: 0x10150
dwc-mipi-csi2-host 4ae00000.csi: gasket: CSI REG: 0x22c1c
dwc-mipi-csi2-host 4ae00000.csi: gasket: MIPI -> ISI pixel ctrl: 0xe07
dwc-mipi-csi2-host 4ae00000.csi: gasket: MIPI -> ISI pixel cnt: 0x0
dwc-mipi-csi2-host 4ae00000.csi: gasket: MIPI -> ISI line cnt: 0x0

~# /unit_tests/memtool 4ae00140 1
E
Reading 0x1 count starting at address 0x4AE00140

0x4AE00140:  00000020

# cat /sys/kernel/debug/clk/cam_pix_root/clk_rate
300000000

from the above result, can see:

1. INT_ST_IPI_FATAL is set, which show fatal IPI interrupt is triggered. 

2. IPI interrupt status register shows FIFO is full (data overflow causes data lose)

Is the result indicates the pixel clock is not fast enough? but I set it already very high (300000000)

/***************************************************************************************
 * camera - CSI@4ae00000 - topaz2m: 0x10@I2C2@44350000
 * 1. power pin:		OSM U6 P0_7
 * 2. reset pin:		OSM U6 P0_6
 * 3. bus-type:			4 (MEDIA_BUS_TYPE_CSI2_DPHY)
 * 4. cfg-clk-range:	28 (24M) https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/ov5640-support-on-i-mx93-with-6-12-20-bsp/ta-p/2129317
 * 5. hs-clk-range:		0x2b (1500M, imx93 RM page 4542)
 * 6. data-lanes:		2
 **************************************************************************************/
set_node_property(&mipi_csi_ep, remote-endpoint, <&edp_from_mipi_csi_topaz2m>);
set_node_property(&mipi_csi_ep, hs-clk-range, <0x2c>);
set_node_property(&mipi_csi, assigned-clock-rates, comma(<300000000>, <24000000>));
&lpi2c2 {
	topaz2m_csi0: topaz2m@10 {
		compatible	= "teledyne,topaz2m";
		reg			= <0x10>;
		lid-gpio	= <&gpioex_osm_u6 7 GPIO_ACTIVE_HIGH>;
		pwdn-gpio	= <&gpioex_osm_u6 6 GPIO_ACTIVE_HIGH>;
		csi_id		= <0>;
		mclk		= <24000000>;
		port {
			edp_from_mipi_csi_topaz2m: endpoint {
				remote-endpoint = <&mipi_csi_ep>;
			};
		};
	};
};

Do you help to share some suggestion so I can go next?

Best regards.

Johnson

[joanxie]

do you mind sharing the dts file with me? let me double check it, and what bsp version do you use?

[zhangjunsheng]

Hi @joanxie ,

      kernel version is 6.6.52.

     In my previous post, I share related dts already. anyway, I share here again.

#include "imx93-11x11-evk.dts"

#define set_node_property(n, p, v) 					n { p = v ; }

/***************************************************************************************
 * camera - CSI@4ae00000 - topaz2m: 0x10@I2C2@44350000
 * 1. power pin:		OSM U6 P0_7
 * 2. reset pin:		OSM U6 P0_6
 * 3. bus-type:			4 (MEDIA_BUS_TYPE_CSI2_DPHY)
 * 4. cfg-clk-range:	28 (24M) https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/ov5640-support-on-i-mx93-with-6-12-20-bsp/ta-p/2129317
 * 5. hs-clk-range:		0x2b (1500M, imx93 RM page 4542)
 * 6. data-lanes:		2
 **************************************************************************************/
set_node_property(&mipi_csi_ep, remote-endpoint, <&edp_from_mipi_csi_topaz2m>);
set_node_property(&mipi_csi_ep, hs-clk-range, <0x2c>);
set_node_property(&mipi_csi, assigned-clock-rates, comma(<300000000>, <24000000>));
&lpi2c2 {
	topaz2m_csi0: topaz2m@10 {
		compatible	= "teledyne,topaz2m";
		reg			= <0x10>;
		lid-gpio	= <&gpioex_osm_u6 7 GPIO_ACTIVE_HIGH>;
		pwdn-gpio	= <&gpioex_osm_u6 6 GPIO_ACTIVE_HIGH>;
		csi_id		= <0>;
		mclk		= <24000000>;
		port {
			edp_from_mipi_csi_topaz2m: endpoint {
				remote-endpoint = <&mipi_csi_ep>;
			};
		};
	};
};

Also there is some patch to CSI/ISI driver. CSI patch is I added for imx93, ISI patch is also used on imx8mp and validate working.

diff --git a/drivers/staging/media/imx/dwc-mipi-csi2.c b/drivers/staging/media/imx/dwc-mipi-csi2.c
index 8a24b0231596..4a29d422d1f1 100644
--- a/drivers/staging/media/imx/dwc-mipi-csi2.c
+++ b/drivers/staging/media/imx/dwc-mipi-csi2.c
@@ -33,12 +33,12 @@
 #include <media/v4l2-device.h>
 #include <media/v4l2-fwnode.h>
 #include <media/v4l2-subdev.h>
-
+#undef dev_dbg
 /*
  * DWC MIPI CSI2 Host registers
  */
 #define DWC_MIPI_CSI2_VERSION			0x0
-
+#define dev_dbg dev_info
 #define DWC_MIPI_CSI2_N_LANES			0x4
 #define DWC_MIPI_CSI2_N_LANES_N_LANES(x)	((x) & 0x7)
 
@@ -401,6 +401,12 @@ static const struct csi2h_pix_format dwc_csi2h_formats[] = {
 	}, {
 		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
 		.fmt_reg = 0x2C,
+	}, {
+		.code = MEDIA_BUS_FMT_Y8_1X8,
+		.fmt_reg = 0x2A,
+	}, {
+		.code = MEDIA_BUS_FMT_Y10_1X10,
+		.fmt_reg = 0x2B,
 	}, {
 		/* sentinel */
 	}
@@ -702,12 +708,14 @@ static void disp_mix_gasket_config(struct dwc_mipi_csi2_host *csi2h)
 		fmt_val = DT_YUV422_8;
 		break;
 	case MEDIA_BUS_FMT_SBGGR8_1X8:
+	case MEDIA_BUS_FMT_Y8_1X8:
 		fmt_val = DT_RAW8;
 		break;
 	case MEDIA_BUS_FMT_SBGGR10_1X10:
 	case MEDIA_BUS_FMT_SGBRG10_1X10:
 	case MEDIA_BUS_FMT_SGRBG10_1X10:
 	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_Y10_1X10:
 		fmt_val = DT_RAW10;
 		break;
 	case MEDIA_BUS_FMT_SBGGR12_1X12:
diff --git a/drivers/staging/media/imx/imx8-isi-cap.c b/drivers/staging/media/imx/imx8-isi-cap.c
index c3d6bbeed6a2..ddcac91c84e2 100644
--- a/drivers/staging/media/imx/imx8-isi-cap.c
+++ b/drivers/staging/media/imx/imx8-isi-cap.c
@@ -54,7 +54,19 @@ struct mxc_isi_fmt mxc_isi_src_formats[] = {
 		.memplanes	= 1,
 		.colplanes	= 1,
 		.align		= 2,
-	}
+	}, {
+		.name		= "Grey8 (Y8)",
+		.fourcc		= V4L2_PIX_FMT_GREY,
+		.depth		= { 8 },
+		.memplanes	= 1,
+		.colplanes	= 1,
+	}, {
+		.name		= "Grey10 (Y10)",
+		.fourcc		= V4L2_PIX_FMT_Y10,
+		.depth		= { 16 },
+		.memplanes	= 1,
+		.colplanes	= 1,
+ 	},
 };
 
 struct mxc_isi_fmt *mxc_isi_get_format(unsigned int index)
@@ -100,6 +112,10 @@ struct mxc_isi_fmt *mxc_isi_get_src_fmt(struct v4l2_subdev_format *sd_fmt)
 	    sd_fmt->format.code == MEDIA_BUS_FMT_UYVY8_1X16||
 	    sd_fmt->format.code == MEDIA_BUS_FMT_YUYV8_2X8)
 		index = 1;
+	else if (sd_fmt->format.code == MEDIA_BUS_FMT_Y8_1X8)
+		index = 2;
+	else if (sd_fmt->format.code == MEDIA_BUS_FMT_Y10_1X10)
+		index = 3;
 	else
 		index = 0;
 	return &mxc_isi_src_formats[index];
@@ -752,7 +768,7 @@ static int mxc_isi_capture_release(struct file *file)
 	struct vb2_queue *q = vdev->queue;
 	int ret = -1;
 
-	if (!isi_cap->is_link_setup)
+	if (!isi_cap->is_link_setup || atomic_read(&mxc_isi->usage_count) == 0)
 		return 0;
 
 	if (isi_cap->is_streaming[isi_cap->id])
@@ -961,6 +977,10 @@ static int mxc_isi_source_fmt_init(struct mxc_isi_cap_dev *isi_cap)
 	src_fmt.pad = source_pad->index;
 	src_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	src_fmt.format.code = MEDIA_BUS_FMT_UYVY8_1X16;
+	bool is_rgb(u32 pix_fmt);
+	bool is_yuv(u32 pix_fmt);
+	if (!is_rgb(dst_f->fmt->fourcc) && !is_yuv(dst_f->fmt->fourcc))
+		src_fmt.format.code = dst_f->fmt->mbus_code;
 	src_fmt.format.width = dst_f->width;
 	src_fmt.format.height = dst_f->height;
 	ret = v4l2_subdev_call(src_sd, pad, set_fmt, NULL, &src_fmt);
@@ -1693,6 +1713,7 @@ static int mxc_isi_register_cap_device(struct mxc_isi_cap_dev *isi_cap,
 		goto err_ctrl_free;
 
 	vdev->ctrl_handler = &isi_cap->ctrls.handler;
+	v4l2_dev->ctrl_handler = &isi_cap->ctrls.handler;
 	v4l2_info(v4l2_dev, "Registered %s as /dev/%s\n",
 		  vdev->name, video_device_node_name(vdev));
 
@@ -1924,3 +1945,9 @@ MODULE_DESCRIPTION("IMX8 Image Sensor Interface Capture driver");
 MODULE_LICENSE("GPL");
 MODULE_ALIAS("ISI Capture");
 MODULE_VERSION("1.0");
+
+// patch
+// line 57-69:		mxc_isi_src_formats[], add 2 formats for topaz2m camera
+// line 115-118:	mxc_isi_get_src_fmt(), add supporting 2 formats for topaz2m camera
+// line 980-983:	mxc_isi_source_fmt_init(), supporting get format from mbus_code
+// line 1716:		mxc_isi_register_cap_device(), pass ctrl handler to v4l2_dev
diff --git a/drivers/staging/media/imx/imx8-isi-fmt.c b/drivers/staging/media/imx/imx8-isi-fmt.c
index db11c809a3d3..16a6317b87e2 100644
--- a/drivers/staging/media/imx/imx8-isi-fmt.c
+++ b/drivers/staging/media/imx/imx8-isi-fmt.c
@@ -97,7 +97,26 @@ struct mxc_isi_fmt mxc_isi_out_formats[] = {
 		.colplanes	= 1,
 		.align		= 2,
 		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
-	}
+	}, {
+		.name		= "Grey8 (Y8)",
+		.fourcc		= V4L2_PIX_FMT_GREY,
+		.depth		= { 8 },
+		.color		= MXC_ISI_OUT_FMT_RAW8,
+		.memplanes	= 1,
+		.colplanes	= 1,
+		.mbus_code	= MEDIA_BUS_FMT_Y8_1X8,
+	}, {
+		.name		= "Grey10 (Y10)",
+		.fourcc		= V4L2_PIX_FMT_Y10,
+		.depth		= { 16 },
+		.color		= MXC_ISI_OUT_FMT_RAW10,
+		.memplanes	= 1,
+		.colplanes	= 1,
+		.mbus_code	= MEDIA_BUS_FMT_Y10_1X10,
+ 	},
 };
 
 size_t mxc_isi_out_formats_size = ARRAY_SIZE(mxc_isi_out_formats);
+
+// patch
+// line 100-116:		mxc_isi_out_formats[], add 2 formats for topaz2m camera
diff --git a/drivers/staging/media/imx/imx8-isi-hw.c b/drivers/staging/media/imx/imx8-isi-hw.c
index f3b2b98798be..cee34d389309 100644
--- a/drivers/staging/media/imx/imx8-isi-hw.c
+++ b/drivers/staging/media/imx/imx8-isi-hw.c
@@ -103,7 +103,7 @@ static void printk_pixelformat(char *prefix, int val)
 		(val >> 24) & 0xff);
 }
 
-static bool is_rgb(u32 pix_fmt)
+bool is_rgb(u32 pix_fmt)
 {
 	if ((pix_fmt == V4L2_PIX_FMT_RGB565) ||
 	    (pix_fmt == V4L2_PIX_FMT_RGB24)  ||
@@ -120,7 +120,7 @@ static bool is_rgb(u32 pix_fmt)
 		return false;
 }
 
-static bool is_yuv(u32 pix_fmt)
+bool is_yuv(u32 pix_fmt)
 {
 	if ((pix_fmt == V4L2_PIX_FMT_YUYV)  ||
 	    (pix_fmt == V4L2_PIX_FMT_YUV32) ||
@@ -812,3 +812,7 @@ void mxc_isi_m2m_start_read(struct mxc_isi_dev *mxc_isi)
 	writel(val, mxc_isi->regs + CHNL_MEM_RD_CTRL);
 }
 EXPORT_SYMBOL_GPL(mxc_isi_m2m_start_read);
+
+// patch
+// line 106:	is_rgb() discard static declaration so can used outside.
+// line 123:	is_yuv() discard static declaration so can used outside.

Best regards.

Johnson

[joanxie]

sorry for my delay, there is some issue with community, I missed the message, do you fix this issue? 

[zhangjunsheng]

Hi @joanxie ,

     Not yet.

     Limited for the condition, I have only the data sheet of the camera, no reference manual of the camera and no high speed oscilloscope to measure the actual frequency of the lanes. from the datasheet, the lane speed is 1.2G. so for 2 lanes and RAW8 case, the pixel speed might be 1.2G x 2 / 8 = 300M, is it right?

    If it's the case, how do I need to configure CSI/ISP clock to avoid the FIFO overflow?

Best regards.

Johnson