topic Re: i.mx8m plus 12 bit grayscale camera iterface issue in i.MX Processors

i.mx8m plus 12 bit grayscale camera iterface issue

tdave — Fri, 10 Nov 2023 22:41:03 GMT

Hello,

We are trying to interface a 3rd party camera sensor configured for12 bit grayscale output with resolution of 640x480. However when trying to capture the raw image through v4l2-ctl, we are not seeing expected data in the captured frame.

We made changes suggested in https://community.nxp.com/t5/i-MX-Processors/How-to-set-up-the-MIPI-CSI-2-and-ISI-to-transfer-RAW10-images-on/td-p/1134413. However we are still not seeing a valid camera output through v4l2-ctl.

Below are the changes that we have made

Added the below format to mxc_isi_src_formats[]

{ .name = "Y12 (Grey)", .fourcc = V4L2_PIX_FMT_Y12, .depth = { 12 }, .memplanes = 1, .colplanes = 1, .align = 1, } struct mxc_isi_fmt *mxc_isi_get_src_fmt(struct v4l2_subdev_format *sd_fmt) sd_fmt->format.code == MEDIA_BUS_FMT_UYVY8_2X8 || sd_fmt->format.code == MEDIA_BUS_FMT_YUYV8_2X8) index = 1; + else if (sd_fmt->format.code == MEDIA_BUS_FMT_Y12_1X12) + index = 2; else index = 0;

Added below format to mxc_isi_out_formats[]

{ .name = "Y12", .fourcc = V4L2_PIX_FMT_Y12, .depth = { 12 }, .color = MXC_ISI_OUT_FMT_RAW12, .memplanes = 1, .colplanes = 1, .align = 1, .mbus_code = MEDIA_BUS_FMT_Y12_1X12, }

Set the ISI CSC in bypass mode:

// if (is_yuv(src_fmt->fourcc) && is_rgb(dst_fmt->fourcc)) { // /* YUV2RGB */ // csc = YUV2RGB; // /* YCbCr enable??? */ // val |= (CHNL_IMG_CTRL_CSC_MODE_YCBCR2RGB << CHNL_IMG_CTRL_CSC_MODE_OFFSET); // val |= (CHNL_IMG_CTRL_YCBCR_MODE_ENABLE << CHNL_IMG_CTRL_YCBCR_MODE_OFFSET); // } else if (is_rgb(src_fmt->fourcc) && is_yuv(dst_fmt->fourcc)) { // /* RGB2YUV */ // csc = RGB2YUV; // val |= (CHNL_IMG_CTRL_CSC_MODE_RGB2YCBCR << CHNL_IMG_CTRL_CSC_MODE_OFFSET); // } else { // /* Bypass CSC */ pr_info("bypass csc\n"); mxc_isi->cscen = 0; val |= CHNL_IMG_CTRL_CSC_BYPASS_ENABLE;

Add the Y12 format to imx8-mipi-csi2-sam.c

static int csis_s_fmt(struct v4l2_subdev *sd, struct csi_sam_format *fmt) case V4L2_PIX_FMT_SRGGB12: code = MEDIA_BUS_FMT_SRGGB12_1X12; break; + case V4L2_PIX_FMT_Y12: + code = MEDIA_BUS_FMT_Y12_1X12; + break; default: return -EINVAL;

v4l2-ctl command to capture the camera frame:

v4l2-ctl --verbose --device /dev/video2 --stream-mmap --stream-to=frame.raw --stream-count=1 --set-fmt-video=width=640,height=480,pixelformat='Y12 '

ISI register dump during the capture:

[ 38.142066] mxc-isi 32e00000.isi: ISI CHNLC register dump, isi0 [ 38.148041] mxc-isi 32e00000.isi: CHNL_CTRL[0x00]: e0000000 [ 38.154595] mxc-isi 32e00000.isi: CHNL_IMG_CTRL[0x04]: e000001 [ 38.161123] mxc-isi 32e00000.isi: CHNL_OUT_BUF_CTRL[0x08]: 7c707 [ 38.167415] mxc-isi 32e00000.isi: CHNL_IMG_CFG[0x0c]: 1e00280 [ 38.173874] mxc-isi 32e00000.isi: CHNL_IER[0x10]: 3cfc0000 [ 38.180420] mxc-isi 32e00000.isi: CHNL_STS[0x14]: 100 [ 38.186540] mxc-isi 32e00000.isi: CHNL_SCALE_FACTOR[0x18]: 10001000 [ 38.193090] mxc-isi 32e00000.isi: CHNL_SCALE_OFFSET[0x1c]: 00 [ 38.199115] mxc-isi 32e00000.isi: CHNL_CROP_ULC[0x20]: 00 [ 38.205142] mxc-isi 32e00000.isi: CHNL_CROP_LRC[0x24]: 00 [ 38.211169] mxc-isi 32e00000.isi: CHNL_CSC_COEFF0[0x28]: 00 [ 38.217198] mxc-isi 32e00000.isi: CHNL_CSC_COEFF1[0x2c]: 00 [ 38.223222] mxc-isi 32e00000.isi: CHNL_CSC_COEFF2[0x30]: 00 [ 38.229249] mxc-isi 32e00000.isi: CHNL_CSC_COEFF3[0x34]: 00 [ 38.235274] mxc-isi 32e00000.isi: CHNL_CSC_COEFF4[0x38]: 00 [ 38.241302] mxc-isi 32e00000.isi: CHNL_CSC_COEFF5[0x3c]: 00 [ 38.247325] mxc-isi 32e00000.isi: CHNL_ROI_0_ALPHA[0x40]: 00 [ 38.253351] mxc-isi 32e00000.isi: CHNL_ROI_0_ULC[0x44]: 00 [ 38.259378] mxc-isi 32e00000.isi: CHNL_ROI_0_LRC[0x48]: 00 [ 38.265412] mxc-isi 32e00000.isi: CHNL_ROI_1_ALPHA[0x4c]: 00 [ 38.271440] mxc-isi 32e00000.isi: CHNL_ROI_1_ULC[0x50]: 00 [ 38.277465] mxc-isi 32e00000.isi: CHNL_ROI_1_LRC[0x54]: 00 [ 38.283491] mxc-isi 32e00000.isi: CHNL_ROI_2_ALPHA[0x58]: 00 [ 38.289514] mxc-isi 32e00000.isi: CHNL_ROI_2_ULC[0x5c]: 00 [ 38.295541] mxc-isi 32e00000.isi: CHNL_ROI_2_LRC[0x60]: 00 [ 38.301587] mxc-isi 32e00000.isi: CHNL_ROI_3_ALPHA[0x64]: 00 [ 38.307627] mxc-isi 32e00000.isi: CHNL_ROI_3_ULC[0x68]: 00 [ 38.313655] mxc-isi 32e00000.isi: CHNL_ROI_3_LRC[0x6c]: 00 [ 38.319683] mxc-isi 32e00000.isi: CHNL_OUT_BUF1_ADDR_Y[0x70]: 84500000 [ 38.326251] mxc-isi 32e00000.isi: CHNL_OUT_BUF1_ADDR_U[0x74]: 00 [ 38.332288] mxc-isi 32e00000.isi: CHNL_OUT_BUF1_ADDR_V[0x78]: 00 [ 38.338323] mxc-isi 32e00000.isi: CHNL_OUT_BUF_PITCH[0x7c]: 3c0 [ 38.344437] mxc-isi 32e00000.isi: CHNL_IN_BUF_ADDR[0x80]: 00 [ 38.350482] mxc-isi 32e00000.isi: CHNL_IN_BUF_PITCH[0x84]: 00 [ 38.356517] mxc-isi 32e00000.isi: CHNL_MEM_RD_CTRL[0x88]: 00 [ 38.362569] mxc-isi 32e00000.isi: CHNL_OUT_BUF2_ADDR_Y[0x8c]: 84100000 [ 38.369122] mxc-isi 32e00000.isi: CHNL_OUT_BUF2_ADDR_U[0x90]: 00 [ 38.375149] mxc-isi 32e00000.isi: CHNL_OUT_BUF2_ADDR_V[0x94]: 00 [ 38.381177] mxc-isi 32e00000.isi: CHNL_SCL_IMG_CFG[0x98]: 1e00280 [ 38.387638] mxc-isi 32e00000.isi: CHNL_FLOW_CTRL[0x9c]: 00

dmesg logs showing camera driver linking to mipi and isi

[ 8.149343] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture] [ 8.157345] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0] [ 8.181549] mx8-img-md: created link [test_cameras 0-0030] => [mxc-mipi-csi2.0]

We would appreciate any help or suggestion to resolve this issue.

Thanks,
Tareesh

Re: i.mx8m plus 12 bit grayscale camera iterface issue

joanxie — Mon, 13 Nov 2023 08:03:48 GMT

since imx8mp has ISP which can support raw data directly, why do you need to use ISI? do you want to get rgb output?

Re: i.mx8m plus 12 bit grayscale camera iterface issue

tdave — Mon, 13 Nov 2023 18:04:30 GMT

Hello @joanxie , Thank you for your response.

We are using ISI with our other camera which uses YUV 4:2:0 format and we wanted to keep the same interface for this other camera sensor. Is the raw mode not supported through ISI ? If we have to switch to using ISP, is there any documentation/guide I can refer to?

Thanks,
Tareesh

Re: i.mx8m plus 12 bit grayscale camera iterface issue

joanxie — Tue, 14 Nov 2023 03:19:01 GMT

ISI can support raw data, but couldn't handle this, if you want to use ISI, it's ok, I checked your source code, I think this should be 16

.depth		= { 12 },

Re: i.mx8m plus 12 bit grayscale camera iterface issue

tdave — Tue, 14 Nov 2023 22:36:43 GMT

Hello @joanxie ,

I implemented the change to depth as you suggested but still don't see a valid frame.

.name = "Y12 (Grey)", .fourcc = V4L2_PIX_FMT_Y12, .depth = { 16 }, .memplanes = 1, .colplanes = 1, .align = 1,

.name = "Y12", .fourcc = V4L2_PIX_FMT_Y12, .depth = { 16 }, .color = MXC_ISI_OUT_FMT_RAW12, .memplanes = 1, .colplanes = 1, .align = 1, .mbus_code = MEDIA_BUS_FMT_Y12_1X12,

Once I tested the depth change, I also made the following changes to imx8-mipi-csi2-sam.c

static const struct csis_pix_format mipi_csis_formats[] +{ + .code = MEDIA_BUS_FMT_Y12_1X12, + .fmt_reg = MIPI_CSIS_ISPCFG_FMT_RAW12, + .data_alignment = 16, +}, static void disp_mix_gasket_config(struct csi_state *state) + case MEDIA_BUS_FMT_Y12_1X12: + fmt_val = GASKET_0_CTRL_DATA_TYPE_RAW12; + break; static int csis_s_fmt(struct v4l2_subdev *sd, struct csi_sam_format *fmt) + case V4L2_PIX_FMT_Y12: + code = MEDIA_BUS_FMT_Y12_1X12; + break;

But still no luck with capturing a valid frame

Here is the v4l2-ctl output:

root@test~# v4l2-ctl --verbose --device /dev/video2 --stream-mmap --stream-to=frame.raw --stream-count=1 --set-fmt-video=width=640,height=480,pixelformat='Y12 ' VIDIOC_QUERYCAP: ok VIDIOC_G_FMT: ok VIDIOC_S_FMT: ok Format Video Capture Multiplanar: Width/Height : 640/480 Pixel Format : 'Y12 ' (12-bit Greyscale) Field : None Number of planes : 1 Flags : Colorspace : Raw Transfer Function : Default YCbCr/HSV Encoding: ITU-R 601 Quantization : Full Range Plane 0 : Bytes per Line : 1280 Size Image : 614400 VIDIOC_REQBUFS returned 0 (Success) VIDIOC_QUERYBUF returned 0 (Success) VIDIOC_QUERYBUF returned 0 (Success) VIDIOC_QUERYBUF returned 0 (Success) VIDIOC_QUERYBUF returned 0 (Success) VIDIOC_QBUF returned 0 (Success) VIDIOC_QBUF returned 0 (Success) VIDIOC_QBUF returned 0 (Success) VIDIOC_QBUF returned 0 (Success) VIDIOC_STREAMON returned 0 (Success) cap dqbuf: 0 seq: 1 bytesused: 614400 ts: 33.186146 (ts-monotonic, ts-src-eof)

Re: i.mx8m plus 12 bit grayscale camera iterface issue

tdave — Thu, 16 Nov 2023 01:27:10 GMT

Dump of MIPI CSI2 registers:

[ 452.436044] mxc-mipi-csi2.0: --- mipi_csis_s_stream --- [ 452.441362] mxc-mipi-csi2.0: CSIS_VERSION[0]: 0x03060301 [ 452.447413] mxc-mipi-csi2.0: CSIS_CMN_CTRL[4]: 0x00004905 [ 452.453459] mxc-mipi-csi2.0: CSIS_CLK_CTRL[8]: 0x000f0000 [ 452.459509] mxc-mipi-csi2.0: CSIS_INTMSK[10]: 0x0fffff1f [ 452.465708] mxc-mipi-csi2.0: CSIS_INTSRC[14]: 0x00000000 [ 452.471832] mxc-mipi-csi2.0: CSIS_DPHYSTATUS[20]: 0x000000f1 [ 452.477950] mxc-mipi-csi2.0: CSIS_DPHYCTRL[24]: 0x0d800007 [ 452.484064] mxc-mipi-csi2.0: CSIS_DPHYBCTRL_L[30]: 0x000001f4 [ 452.490175] mxc-mipi-csi2.0: CSIS_DPHYBCTRL_H[34]: 0x00000000 [ 452.496288] mxc-mipi-csi2.0: CSIS_DPHYSCTRL_L[38]: 0x00000000 [ 452.502402] mxc-mipi-csi2.0: CSIS_DPHYSCTRL_H[3c]: 0x00000000 [ 452.508513] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH0[40]: 0x000000b0 [ 452.514627] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH1[50]: 0x000008fd [ 452.520734] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH2[60]: 0x000008fe [ 452.526843] mxc-mipi-csi2.0: CSIS_ISPCONFIG_CH3[70]: 0x000008ff [ 452.532951] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH0[44]: 0x01e00280 [ 452.539061] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH1[54]: 0x80008000 [ 452.545169] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH2[64]: 0x80008000 [ 452.551281] mxc-mipi-csi2.0: CSIS_ISPRESOL_CH3[74]: 0x80008000 [ 452.557390] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH0[48]: 0x00000000 [ 452.563507] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH1[58]: 0x00000000 [ 452.569622] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH2[68]: 0x00000000 [ 452.575752] mxc-mipi-csi2.0: CSIS_ISPSYNC_CH3[78]: 0x00000000 [ 452.581875] mxc-mipi-csi2.0: --- mipi_csis_s_stream --- [ 452.587122] mxc-mipi-csi2.0: GPR_GASKET_0_CTRL[60]: 0xffff8000 [ 452.593240] mxc-mipi-csi2.0: GPR_GASKET_0_HSIZE[64]: 0xffff8000 [ 452.599352] mxc-mipi-csi2.0: GPR_GASKET_0_VSIZE[68]: 0xffff8000

Re: i.mx8m plus 12 bit grayscale camera iterface issue

joanxie — Thu, 16 Nov 2023 09:01:22 GMT

你可以参考下下面两个客户的配置，我这里没有raw12的camera可以复现这个问题，首先可以看下你的camera 是什么模式，连续还是非连续的，我看这个客户depth用12也可以成功捕获图像

“https://community.nxp.com/t5/i-MX-Processors/raw12-camera-on-imx8mp-isi/m-p/1653019”

“https://community.nxp.com/t5/i-MX-Processors/imx8mp-Failed-to-obtain-sensor-raw12-data/m-p/1537266”

Re: i.mx8m plus 12 bit grayscale camera iterface issue

tdave — Mon, 20 Nov 2023 23:45:22 GMT

After enabling the debug logs for the MIPI interrupts, I noticed the following in the logs:

[ 475.813567] mxc-mipi-csi2.0: SOT Error: 1 [ 475.817582] mxc-mipi-csi2.0: Lost Frame Start Error: 1 [ 475.822723] mxc-mipi-csi2.0: status: 00031000

We have the following values in the device tree:

data-lanes = <2>; csis-hs-settle = <13>; csis-clk-settle = <2>;

@joanxie I would appreciate if you could give some suggestion as to what might be wrong in the configuration to cause the start of frame to be lost.

Thanks.

Re: i.mx8m plus 12 bit grayscale camera iterface issue

joanxie — Thu, 23 Nov 2023 03:42:35 GMT

it seems that the timing issue, you can refer to the chapter 3.2.5 DPHY high-speed settle timer of enclosed file

Re: i.mx8m plus 12 bit grayscale camera iterface issue

tdave — Thu, 30 Nov 2023 05:53:22 GMT

I have set the value based on the document. But still don't see any valid frame output.

csis-hs-settle = <17>;
csis-clk-settle = <2>;

Thanks,
Tareesh

Re: i.mx8m plus 12 bit grayscale camera iterface issue

joanxie — Mon, 04 Dec 2023 08:53:19 GMT

did you set this based on your measured clock by Oscilloscope? if no, try to measure it