IMX8MM Camera IMX298 module bringup

[mohamed_gaseen]

Hi,

We are currently working on the IMX8M-MINI board for bringing up the camera module IMX298.

Here the issue is that the RAW10 format is  not supported in IMX8MM MIPI-CSI capture driver.

Is it possible?

Can you please share me, how to add the RAW10 patches in the MIPI-CSI capture driver?

IMX298 support RAW8/10, Our driver has been configured with the below format and tried to stream the camera but it got failed.

MEDIA_BUS_FMT_SRGGB10_1X10 => MIPI-CSI capture driver doesn't have the configuration

MEDIA_BUS_FMT_SBGGR8_1X8 => MIPI-CSI capture driver support this format

In my imx298 driver code, we have tried to configure in RAW8 in the mbus_code "MEDIA_BUS_FMT_SBGGR8_1X8" but we are failed to stream the camera.

Can you please share the procedure to capture the raw image in a specific format RAW8/RAW10 and include the addition of RAW10 patches in the IMX8MM MIPI-CSI capture driver?

Thanks in Advance.

[mohamed_gaseen]

Hi,

   Is there any difference about BGGR and RGGB format because of the patch configured the BGGR format. Our camera driver is configured to rggb can you please add related to the color format patch

[joanxie]

if you don't use the same bayer pattern, you need change this register

[santhana_kumar]

We are trying to stream the camera in RGGB format  & getting the interrupt of CSI but not the raw image is captured using the below command

gst-launch-1.0 -vvv v4l2src device=/dev/video0 num-buffers=1 \! "video/x-bayer,width=1476,height=834,framerate=(fraction)25/1,format=(string)rggb" \! bayer2rgb ! pngenc ! filesink location=test.png

Below changes for RGGB
in the function changed as per your suggestions
csi_dmareq_rff_enable()
//cr2 |= 0xC0000000;
cr2 |= 0xC0080000;


[ 68.921246] Zumi CAPTURE warn mx6s_csi_irq_handler Rx fifo overflow
[ 69.673795] Zumi CAPTURE warn mx6s_csi_irq_handler Rx fifo overflow
[ 69.680190] Zumi CAPTURE warn mx6s_csi_irq_handler Rx fifo overflow
[ 70.432761] Zumi CAPTURE warn mx6s_csi_irq_handler Rx fifo overflow
[ 70.439151] Zumi CAPTURE warn mx6s_csi_irq_handler Rx fifo overflow
root@imx8mmevk:~# cat /proc/interrupts | grep csi
56: 81 0 0 0 GICv3 48 Level csi
57: 9266 0 0 0 GICv3 49 Level 32e30000.mipi_csi
root@imx8mmevk:~# cat /proc/interrupts | grep csi
56: 84 0 0 0 GICv3 48 Level csi
57: 9391 0 0 0 GICv3 49 Level 32e30000.mipi_csi
root@imx8mmevk:~# cat /proc/interrupts | grep csi
56: 87 0 0 0 GICv3 48 Level csi
57: 9806 0 0 0 GICv3 49 Level 32e30000.mipi_csi
root@imx8mmevk:~#

[mohamed_gaseen]

Thanks for replying back. 
Please check the below registers want to change 
#define CSI_CSICR2 0x4

After adding the patch using the gstreamer command to stream the camera below logs are attached.

The below issue of running RAW10

SBGGR8 => It calls the stream function

SRGGB10 => It doesn't calls the stream function 

My driver was called imx298_enum_mbus_code but not the imx298_enum_frame_size 

[joanxie]

sent message to you, pls check

[santhana_kumar]

Joan,
Can you reply this below and support the issue

Using v4l2-ctl command to stream the camera below the logs
v4l2-ctl --device=/dev/video0 --set-fmt-video=width=1476,height=834,pixelformat=RG10 --stream-mmap --stream-count=1 --stream-to=output.raw

 CAPTURE warn mx6s_csi_irq_handler Rx fifo overflow
 mx6s_csi_irq_handler 1176 cr19 = 0xff

[joanxie]

sorry, I take annual leave this week, since I couldn't reproduce this issue and other customer can set raw10 on im8mm without any fifo overflow issue, I only can give you some suggestions

1) you should already notice this link, you can try to change the register as the the internal team said

https://community.nxp.com/t5/i-MX-Processors/RX-fifo-overflow-on-MIPI-CSI2-i-MX8MQ/m-p/1087696

2) try to change the data lane to 2

[santhana_kumar]

1. Lane Changed :

We are tried changes from lane 4 to 2 but still the issue RX FIFO overflow came.

2. Go through the RX FIFO link and sees the patches but it seems the imx8mq. Can you please sent the imx8mm patch files regarding the RX fifo overflow issue?

[joanxie]

imx8mm and imx8mq uses the same csi ip, you can find they use the same csi driver,  try to change the registers as link mentions, no finding other patches for this issue

[santhana_kumar]

Hi,

  1. Are you telling the below patch to be apply in the IMX8MM board?

2. If this is the patch,  IMX8MM driver is not using the below file
drivers/media/platform/imx8/mxc-mipi-csi2_yav.c

3. Can you please confirm the below changes for the RGGB 

Filename: drivers/media/platform/mxc/capture/mx6s_capture.c

API: csi_dmareq_rff_enable

Code changes:
/* Burst Type of DMA Transfer from RxFIFO. INCR16 */
+ //cr2 |= 0xC0000000;
+ cr2 |= 0xC0080000;

Please confirm the above points which patch will resolve the CSI RXFIFO overflow error?

[joanxie]

1)I'm not saying this patch, I mean you can change the csi phy register when you get the wrong value from CSICR19, if you get overflow error, try the stpes: 1)Disable the CSI, 2)Clear the RX FIFO,3) Reflash the DMA controller, 4) then enable CSI

2)try to change the cr2 register under function "mx6s_configure_csi", for example

csi_dev->fmt->pixelformat == BIT_MIPI_DATA_FORMAT_RAW10) {
cr2 |= (0x2 << 19);

csi_write(csi_dev, cr2, CSI_CSICR2);}

this is just an example, pls change it according to your camera

[santhana_kumar]

Joan,

Previously we faced the RXFIFO overflow error occurred in the mipi-csi driver.
Now the below logs saying that the CRC and ECC error event occured.

mx6s_csi_irq_handler CSI Status: 0xffffffff80014000
[ 40.510745] mx6s_csi_irq_handler CSI cr3 = 0x000010a8 cr18 = 0xd65ad030 cr19 = 0x00000000
[ 40.884287] mx6s_csi_irq_handler CSI Status: 0xffffffff80014001
[ 40.890214] mx6s_csi_irq_handler CSI cr3 = 0x000110a8 cr18 = 0xd65ad030 cr19 = 0x00000010
[ 41.263750] mx6s_csi_irq_handler CSI Status: 0xffffffff92034001
[ 41.269672] mx6s_csi_irq_handler CSI cr3 = 0x000210a8 cr18 = 0xd65ad030 cr19 = 0x00000010
[ 41.277848]  base address switching Change Err.

[ 45.288008] mxc_mipi-csi.0: Frame End events: 13
[ 45.292625] mxc_mipi-csi.0: Frame Start events: 13
[ 45.297413] mxc_mipi-csi.0: Non-image data after odd frame events: 0
[ 45.303765] mxc_mipi-csi.0: Non-image data before odd frame events: 0
[ 45.310202] mxc_mipi-csi.0: Non-image data after even frame events: 0
[ 45.316639] mxc_mipi-csi.0: Non-image data before even frame events: 0
[ 45.323166] mxc_mipi-csi.0: Unknown Error events: 0
[ 45.328041] mxc_mipi-csi.0: CRC Error events: 5
[ 45.332568] mxc_mipi-csi.0: ECC Error events: 3
[ 45.337096] mxc_mipi-csi.0: FIFO Overflow Error events: 0
[ 45.342492] mxc_mipi-csi.0: Lost Frame End Error events: 0
[ 45.347974] mxc_mipi-csi.0: Lost Frame Start Error events: 0
[ 45.353630] mxc_mipi-csi.0: SOT Error events: 0

In the mx6_capture.c file the MXC_MIPI_CSI ocuured the CSI error BIT_ADDR_CH_ERR_INT, Can you please support us to resolve the issue in IMX8M-MINI evk board?

[joanxie]

I don't know what your changing caused this issue, if this isn't related to original issue, you'd better create a new thread and put all of detailed information there, The CSI module supports so-called double buffer mode, in which it can store two consecutive frames into  different frame buffers to simplify further frame processing in software. we suggest customer to set BASEADDR_SWITCH_EN = 1 and BASEADDR_SWITCH_SEL=1, you can check these registers

[santhana_kumar]

Joan,

CR18 register already sets the bit value 1 in the bits (BASEADDR_SWITCH_EN & BASEADDR_SWITCH_SEL).

Below logs are attached:
[ 111.933056] set to pixelformat 'RAWRGB'
[ 111.936932] type 1
[ 111.938945] width 1476
[ 111.941393] height 834
[ 111.943771] Zumi In function: mx6s_configure_csi - pixelformat 0x30314752
[ 111.950589] Zumi csi_set_imagpara Before width = 1476
[ 111.955844] Zumi csi_set_imagpara After width = 5904
[ 111.980236] Zumi __mipi_csis_set_format fmt: 0x300f, 1476 x 834

[ 113.463804] IMX298: stream on
[ 113.845040] mx6s_csi_irq_handler CSI csr_val = 0x80014000
[ 113.850444] mx6s_csi_irq_handler CSI cr1 = 0x011b0903
[ 113.855493] mx6s_csi_irq_handler CSI cr2 = 0xc0080000
[ 113.860541] mx6s_csi_irq_handler CSI cr3 = 0x000010a8
[ 113.865592] mx6s_csi_irq_handler CSI cr18 = 0xd65ad030
[ 113.870727] mx6s_csi_irq_handler CSI cr19 = 0x00000000
[ 113.875864] mx6s_csi_irq_handler CSI imgpara = 0x17100342
[ 114.224519] mx6s_csi_irq_handler CSI csr_val = 0x80014001
[ 114.229925] mx6s_csi_irq_handler CSI cr1 = 0x011b0903
[ 114.234973] mx6s_csi_irq_handler CSI cr2 = 0xc0080000
[ 114.240021] mx6s_csi_irq_handler CSI cr3 = 0x000110a8
[ 114.245073] mx6s_csi_irq_handler CSI cr18 = 0xd65ad030
[ 114.250208] mx6s_csi_irq_handler CSI cr19 = 0x00000010
[ 114.255342] mx6s_csi_irq_handler CSI imgpara = 0x17100342
[ 114.603971] mx6s_csi_irq_handler CSI csr_val = 0x92034001
[ 114.609375] mx6s_csi_irq_handler CSI cr1 = 0x011b0903
[ 114.614423] mx6s_csi_irq_handler CSI cr2 = 0xc0080000
[ 114.619472] mx6s_csi_irq_handler CSI cr3 = 0x000210a8
[ 114.624521] mx6s_csi_irq_handler CSI cr18 = 0xd65ad030
[ 114.629656] mx6s_csi_irq_handler CSI cr19 = 0x00000010
[ 114.634791] mx6s_csi_irq_handler CSI imgpara = 0x17100342
[ 114.640187] base address switching Change Err.

After stream off below the logs are captured from the CSI events

[ 137.042608] IMX298: stream off
[ 137.045693] mxc_mipi-csi.0: Frame End events: 62
[ 137.050308] mxc_mipi-csi.0: Frame Start events: 62
[ 137.055096] mxc_mipi-csi.0: Non-image data after odd frame events: 0
[ 137.061446] mxc_mipi-csi.0: Non-image data before odd frame events: 0
[ 137.067881] mxc_mipi-csi.0: Non-image data after even frame events: 0
[ 137.074317] mxc_mipi-csi.0: Non-image data before even frame events: 0
[ 137.080842] mxc_mipi-csi.0: Unknown Error events: 0
[ 137.085716] mxc_mipi-csi.0: CRC Error events: 29
[ 137.090328] mxc_mipi-csi.0: ECC Error events: 15
[ 137.094943] mxc_mipi-csi.0: FIFO Overflow Error events: 0
[ 137.100337] mxc_mipi-csi.0: Lost Frame End Error events: 0
[ 137.105818] mxc_mipi-csi.0: Lost Frame Start Error events: 0
[ 137.111473] mxc_mipi-csi.0: SOT Error events: 0

[joanxie]

try to this patch

static int mx6s_csi_enable(struct mx6s_csi_dev *csi_dev)

if (pix->field == V4L2_FIELD_INTERLACED)
csi_tvdec_enable(csi_dev, true);
+ else
+ csi_tvdec_enable(csi_dev, false);

/* For mipi csi input only */
if (csi_dev->csi_mipi_mode == true) {

for the FIFO overflow issue, you can refer to the workaround of ERR050384: MIPI CSI: Receive FIFO Overflow may lead to system hang, I attached the errata

[santhana_kumar]

We are already applied the patch and we can able to capture the image
For RAW10 image :
If the input data from the sensor is 10-bit/pixel or 16-bit/pixel format, the IMAGE_WIDTH should be a multiple of 4 pixels.

[santhana_kumar]

Joan,

Disabled the CR18 bits(BASEADDR_SWITCH_EN & BASEADDR_SWITCH_SEL) and now we are able to capture the image. please see the attached picture

Can you please explain detail why if we can disable the bit and we can able to capture the image?

[joanxie]

in fact, you should get detailed register description in the reference manual, BASEADDR_SWITCH_EN controls the buffer start address switch when one frame image data is done. When this bit is set to 1'b0, the start address of frame buffer will not change, and the image data is store in one buffer so If set BASEADDR_SWITCH_EN = 0, not care BASEADDR_SWITCH_SEL. The frame buffer only used one and the data is refresh by the embedded DMA.

[santhana_kumar]

Thank you, Joan.

Could you please share the patch for enabling RAW image capture in the Android layer? We are facing issues while capturing RAW images from the Android layer.

We followed the steps in the link below and applied only the patch for the HAL. However, we encountered issues with the Java files. If you could share the complete patch file, it would be greatly appreciated.

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX8MP-support-raw-format-on-Android/ta...

[santhana_kumar]

Joan,

Connecting the IMX298 camera module to the IMX8MM evk board we have an issue in the below lines.
1. CSI RX FIFO overflow occured
2. How to check my IMX8MM configured as MIPI-CSI2 interface? Observed the drver difference between the IMX8QXP and IMX8MM.

Next step:
We are connected the IMX298 module to the IMX8QXP board.
Added the RAW10 patches on my own in the kernel BSP 5.4.24.
Able to capture the stream and got the ISI interrupt and raw image file also captured.
Attached the raw file there is data available but only the lines are displayed.
Using the below command
v4l2-ctl --device=/dev/video0 --set-fmt-video=width=1476,height=834,pixelformat=RG10 --stream-mmap --stream-count=1 --stream-to=output.raw

Please give us the support to stream the camera in at-least in the IMX8QXP platform?
@joanxie