IMX8 MIPI CSI2 video capture not working

[prasad_imx8]

Hello Community! 

we have a custom design unfortunately with Imx8qxp Rev B0. 

Our video IN pipe line looks : 

1280x800 @60 fps over gmsl----->MAX9288--->IMX8 Mipi CSI 2

I am sure of the setting in the MAX988 serializer however not sure of the settings on the MIPI CSI and ISI configuration. The observation is I dont get any interrupt from the ISI port. I read the registers form the CSI and ISI they read as follows:

mxc-isi 58100000.isi: ISI CHNLC register dump, isi0
mxc-isi 58100000.isi: CHNL_CTRL 0x0h = 0xe0ff0002
mxc-isi 58100000.isi: CHNL_IMG_CTRL 0x4h = 0x20000001
mxc-isi 58100000.isi: CHNL_OUT_BUF_CTRL 0x8h = 0x c000
mxc-isi 58100000.isi: CHNL_IMG_CFG 0xCh = 0x 3200500
mxc-isi 58100000.isi: CHNL_IER 0x10h = 0x3dff0000
mxc-isi 58100000.isi: CHNL_STS 0x14h = 0x 200
mxc-isi 58100000.isi: CHNL_SCALE_FACTOR 0x18h = 0x10001000
mxc-isi 58100000.isi: CHNL_SCALE_OFFSET 0x1Ch = 0x 0
mxc-isi 58100000.isi: CHNL_CROP_ULC 0x20h = 0x 0
mxc-isi 58100000.isi: CHNL_CROP_LRC 0x24h = 0x 0
mxc-isi 58100000.isi: CHNL_CSC_COEFF0 0x28h = 0x 0
mxc-isi 58100000.isi: CHNL_CSC_COEFF1 0x2Ch = 0x 0
mxc-isi 58100000.isi: CHNL_CSC_COEFF2 0x30h = 0x 0
mxc-isi 58100000.isi: CHNL_CSC_COEFF3 0x34h = 0x 0
mxc-isi 58100000.isi: CHNL_CSC_COEFF4 0x38h = 0x 0
mxc-isi 58100000.isi: CHNL_CSC_COEFF5 0x3Ch = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_0_ALPHA 0x40h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_0_ULC 0x44h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_0_LRC 0x48h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_1_ALPHA 0x4Ch = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_1_ULC 0x50h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_1_LRC 0x54h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_2_ALPHA 0x58h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_2_ULC 0x5Ch = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_2_LRC 0x60h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_3_ALPHA 0x64h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_3_ULC 0x68h = 0x 0
mxc-isi 58100000.isi: CHNL_ROI_3_LRC 0x6Ch = 0x 0
mxc-isi 58100000.isi: CHNL_OUT_BUF1_ADDR_Y 0x70h = 0xa6700000
mxc-isi 58100000.isi: CHNL_OUT_BUF1_ADDR_U 0x74h = 0x 0
mxc-isi 58100000.isi: CHNL_OUT_BUF1_ADDR_V 0x78h = 0x 0
mxc-isi 58100000.isi: CHNL_OUT_BUF_PITCH 0x7Ch = 0x a00
mxc-isi 58100000.isi: CHNL_IN_BUF_ADDR 0x80h = 0x 0
mxc-isi 58100000.isi: CHNL_IN_BUF_PITCH 0x84h = 0x 0
mxc-mipi-csi2 58227000.csi: mipi_csi2_s_stream: 1, csi2dev: 0x0

mxc-mipi-csi2 58227000.csi: width=1280, height=800, fmt.code=0x2008
MIPI CSI2 HC register dump, mipi csi0
MIPI CSI2 HC num of lanes 0x100 = 0x3
MIPI CSI2 HC dis lanes 0x104 = 0x0
MIPI CSI2 HC BIT ERR 0x108 = 0x0
MIPI CSI2 HC IRQ STATUS 0x10C = 0x8
MIPI CSI2 HC IRQ MASK 0x110 = 0x1ff
MIPI CSI2 HC ULPS STATUS 0x114 = 0x0
MIPI CSI2 HC DPHY ErrSotHS 0x118 = 0x0
MIPI CSI2 HC DPHY ErrSotSync 0x11c = 0x0
MIPI CSI2 HC DPHY ErrEsc 0x120 = 0x0
MIPI CSI2 HC DPHY ErrSyncEsc 0x124 = 0x0
MIPI CSI2 HC DPHY ErrControl 0x128 = 0x0
MIPI CSI2 HC DISABLE_PAYLOAD 0x12C = 0x0
MIPI CSI2 HC DISABLE_PAYLOAD 0x130 = 0x0
MIPI CSI2 HC IGNORE_VC 0x180 = 0x0
MIPI CSI2 HC VID_VC 0x184 = 0x0
MIPI CSI2 HC FIFO_SEND_LEVEL 0x188 = 0x0
MIPI CSI2 HC VID_VSYNC 0x18C = 0x0
MIPI CSI2 HC VID_SYNC_FP 0x190 = 0x0
MIPI CSI2 HC VID_HSYNC 0x194 = 0x0
MIPI CSI2 HC VID_HSYNC_BP 0x198 = 0x0
MIPI CSI2 CSR register dump
MIPI CSI2 CSR PLM_CTRL 0x000 = 0x801
MIPI CSI2 CSR PHY_CTRL 0x004 = 0x2000bf
MIPI CSI2 CSR PHY_Status 0x008 = 0x1
MIPI CSI2 CSR PHY_Test_Status 0x010 = 0x0
MIPI CSI2 CSR PHY_Test_Status 0x014 = 0x0
MIPI CSI2 CSR PHY_Test_Status 0x018 = 0x0
MIPI CSI2 CSR PHY_Test_Status 0x01C = 0x0
MIPI CSI2 CSR PHY_Test_Status 0x020 = 0x0
MIPI CSI2 CSR VC Interlaced 0x030 = 0x0
MIPI CSI2 CSR Data Type Dis 0x038 = 0x0
MIPI CSI2 CSR 420 1st type 0x040 = 0x0
MIPI CSI2 CSR Ctr_Ck_Rst_Ctr 0x044 = 0x1
MIPI CSI2 CSR Stream Fencing 0x048 = 0x0
MIPI CSI2 CSR Stream Fencing 0x04C = 0x0
mxc-isi 58100000.isi: mxc_isi_pipeline_enable ,entity is no v4l2, mxc_isi.0.capture

Can you guys see if something here is a red flag?  I am not using virtual channel and using 4 lanes of CSI. apart from video capture there is no other application which is using the ISI.

Any help is greatly appreciated.

Best regards,
Prasad.

[malik_cisse]

Hi prasad_imx8,

How do you read ISI registers?

I have tryied to set 
echo -n 'file imx8-isi-hw.c +p' > /sys/kernel/debug/dynamic_debug/control 

but this did not work
Here is what I want to see:

linux-imx/drivers/staging/media/imx/imx8-isi-hw.c

#ifdef DEBUG
void dump_isi_regs(struct mxc_isi_dev *mxc_isi)
{
struct device *dev = &mxc_isi->pdev->dev;
struct {
u32 offset;
const char *const name;
} registers[] = {
{ 0x00, "CHNL_CTRL" },
{ 0x04, "CHNL_IMG_CTRL" },
{ 0x08, "CHNL_OUT_BUF_CTRL" },
{ 0x0C, "CHNL_IMG_CFG" },
{ 0x10, "CHNL_IER" },
{ 0x14, "CHNL_STS" },
{ 0x18, "CHNL_SCALE_FACTOR" },
{ 0x1C, "CHNL_SCALE_OFFSET" },
{ 0x20, "CHNL_CROP_ULC" },
{ 0x24, "CHNL_CROP_LRC" },
{ 0x28, "CHNL_CSC_COEFF0" },
{ 0x2C, "CHNL_CSC_COEFF1" },
{ 0x30, "CHNL_CSC_COEFF2" },
{ 0x34, "CHNL_CSC_COEFF3" },
{ 0x38, "CHNL_CSC_COEFF4" },
{ 0x3C, "CHNL_CSC_COEFF5" },
{ 0x40, "CHNL_ROI_0_ALPHA" },
{ 0x44, "CHNL_ROI_0_ULC" },
{ 0x48, "CHNL_ROI_0_LRC" },
{ 0x4C, "CHNL_ROI_1_ALPHA" },
{ 0x50, "CHNL_ROI_1_ULC" },
{ 0x54, "CHNL_ROI_1_LRC" },
{ 0x58, "CHNL_ROI_2_ALPHA" },
{ 0x5C, "CHNL_ROI_2_ULC" },
{ 0x60, "CHNL_ROI_2_LRC" },
{ 0x64, "CHNL_ROI_3_ALPHA" },
{ 0x68, "CHNL_ROI_3_ULC" },
{ 0x6C, "CHNL_ROI_3_LRC" },
{ 0x70, "CHNL_OUT_BUF1_ADDR_Y" },
{ 0x74, "CHNL_OUT_BUF1_ADDR_U" },
{ 0x78, "CHNL_OUT_BUF1_ADDR_V" },
{ 0x7C, "CHNL_OUT_BUF_PITCH" },
{ 0x80, "CHNL_IN_BUF_ADDR" },
{ 0x84, "CHNL_IN_BUF_PITCH" },
{ 0x88, "CHNL_MEM_RD_CTRL" },
{ 0x8C, "CHNL_OUT_BUF2_ADDR_Y" },
{ 0x90, "CHNL_OUT_BUF2_ADDR_U" },
{ 0x94, "CHNL_OUT_BUF2_ADDR_V" },
{ 0x98, "CHNL_SCL_IMG_CFG" },
{ 0x9C, "CHNL_FLOW_CTRL" },
};

u32 i;

dev_dbg(dev, "ISI CHNLC register dump, isi%d\n", mxc_isi->id);
for (i = 0; i < ARRAY_SIZE(registers); i++) {
u32 reg = readl(mxc_isi->regs + registers[i].offset);
dev_dbg(dev, "%20s[0x%.2x]: %.2x\n",
registers[i].name, registers[i].offset, reg);
}
}

I could set #if 1 and recompile but this is not practical.
Thanks

[yuanke]

Hello everyone,i write driver for max9288 and 96705,it probes ok and config registers ok.But when i test capture using mx8_drm_capture.c ,it blocked at DQBUF,looks like your driver,did you solve the problem?Please help.

[prasad_imx8]

I was able to capture a video stream of 1920x1440 with Imx8qxp sample B0. According to NXP it should not work as per the errata. Now what is the errata? Is it (1920x1440) not supported or is there any other reliability issue. we have some boards with B0 samples. Can we continue to use it. Our customers are furious. We need to tell them why is it working or why they should not use B0 sample going forward.

[prasad_imx8]

Dear Joan,

I was able to test 1920x1440 @30fps with Imx8qxp B0. Now, we want to know why is it working? what is the exact errata? Is it not reliable or not supported at all. Our customers are furious with NXPs Response. We have already some boards mounted with B0. Please let us know what will happen if we continue to use B0 with 1920x1440. What issue can we encounter in future.

Source Code has two Drivers for MIPI CSI2 mxc-mipi-csi2_yav.c and mxc-mipi-csi2.c why two Drivers? The Linux reference Manual lists both. What is the differnce? What should I use? What are the issue we can face using These two Drivers?

Best Regards,

Prasad.

[joanxie]

I post the errata to you:

e50066: ISI: Data overflows occur when input streams exceed AXI transaction
frequency
Description: The Image Sensing Interface (ISI) has a short elasticity buffer relative to the length of a line.
The buffer can be as few as 85 pixels or as many as 512 pixels depending on the output
format. Most RGB formats have 128 pixels. Because of the short buffer, if there is any delay in
latency, then an overflow can occur. The possibility of overflow increases when the number of
active channels increases.
In addition, memory reads and the last line of a scaling process consume data as fast as
possible (instead of at the rate of the incoming pixel stream), therefore, the output buffer fills
faster and requires even lower latency to process the data.
Workaround: The design target was intended to support up to a single 8 Mpixel (4K) stream at 30 fps, or
multiple streams up to the equivalent data rate. However, combinations of sensors which add
up to less than 2Mpixel are supported with current design. That’s to say, if 1 sensor is used,
2Mpixels stream can be supported; if 2 sensors are used, 1Mpixels of each stream can be
supported; and so on.

In the case of scaling, the last line of each frame must be cropped and discarded

refer to the errata, if customer continue using more than 2Mpixel, maybe they will have overflow issue

for mxc-mipi-csi2_yav.c, which is for imx8mq mipi csi driver,

"https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8?h=imx_5.4.70_2..."

mxc-mipi-csi2.c is for imx6 or imx7 in another location

"https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/mxc/mipi?h=imx_5.4.70_2.3.0"

[nagashima]

Hello, Community

Configuration
1280x720@30fps CAM x 4 ---> TP2855 --- YUV422(16bit) / Virtual Channel --> i.MX8QM MIPI-CSI2

I believe the maximum size for RGB is 2MPixel, but is 2MPixel also the maximum size for YUV422 (16bit)?

In the case of RGB format, it is actually ARGB, so it is 32bit, and if 1pixel=8bit, it will use 4 pixels, so it is 128Pixel.

I understand that this issue occurs because it takes time to expand to the output buffer, and this 128pixel buffer is overwritten during that time.

I understand that YUV422(16bit) uses 2Pixel, so it is 256Pixel, but in that case, will it still be the above limit of 2MPixel?

I am running it with the above configuration and have not experienced any symptoms such as the image freezing, but in what specific cases is the likelihood of an overflow occurring?
For example, is it more likely to occur when the chip temperature is high or low? Or is it more likely to occur with RGB format, but less likely to occur with YUV because the 2MPixel limit is relaxed?

[prasad_imx8]

To add we are using linux 4.19.35-1.1.0 which is pretty old. Does this BSP has full support for Imx8 changes for the CSI and ISI module updates (from the previous versions Imx6)? Also the integration of this BSP with V4L2 framework is it complete? 

we cannot test our setup with newer BSP sadly

best regards,
Prasad.

[joanxie]

do you get any error message? how about using the lower resolution, any improvement?

[prasad_imx8]

Dear Joan,

No every thing is good. My application just hangs (waiting for the End of frame interrupt). The buffers are queued. Next the ISR should give the filled buffer back to the application else the application will be hanging.

This is the lowest resolution we can try now because of the limitation from the source we have.

best regards,
Prasad.

[joanxie]

refer to the source code, imx8 supports this already, but what capture format do you use? for imx8qxp B0, refer to the errata：

The design target was intended to support up to a single 8 Mpixel (4K) stream at 30 fps, or
multiple streams up to the equivalent data rate. However, combinations of sensors which add
up to less than 2Mpixel are supported with current design. That’s to say, if 1 sensor is used,
2Mpixels stream can be supported; if 2 sensors are used, 1Mpixels of each stream can be
supported; and so on.
In the case of scaling, the last line of each frame must be cropped and discarded.

so try to update to C0 or use less than 2Mpixel if you use one camera