How to set up the MIPI CSI-2 and ISI to transfer RAW10 images on the i.MX8QM

cancel
Showing results for 
Search instead for 
Did you mean: 

How to set up the MIPI CSI-2 and ISI to transfer RAW10 images on the i.MX8QM

Jump to solution
1,953 Views
gabrielaugustoc
Contributor II

My work is based on Linux Kernel 4.14.170

I have succeeded to stream images through MIPI CSI-2 from a 16Mpx image sensor based on the following article: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX8QXP-capture-raw-bayer-data-and-deba...

Also, I have created the device driver for this image sensor and changed other device drivers into /drivers/media/platform/imx8/ folder.

The ISI has been set to bypass the image format conversion (CSC), when the image sensor is configured as V4L2_PIX_FMT_SGRBG10 (mxc_isi_channel_set_csc function, in the mxc-isi-hw.c file)

The iMX8 is receiving bytes, the buffer size is correct, like for 640x480x10bits = 384000bytes, and it is possible to display the frames from the image sensor but some bytes are missing, like show in the example below: 00 14 00 34 00 14 00 34 00 3C 00 58 00 68 00 68 00 4C 00 44 00 38 00 38 00 60 00 54 00 34 00 54 00 34 00 24 00 6C 00 24 00 70 00 34 00 68 00 28 00 4C 00 48 00 4C 00 18 00 44 00 3C 00 44 00 54

It looks like the MIPI CSI-2 or Pixel Link is not transmitting the RAW10, just 8 bits.

How can I set up the MIPI CSI-2 and/or the Pixel Link for RAW10 bits?

0 Kudos
1 Solution
1,886 Views
gabrielaugustoc
Contributor II

Hi everyone,

I got the solution for this issue:

{
.name = "BA10",
.fourcc = V4L2_PIX_FMT_SGRBG10,
.depth = { 16 },
.color = MXC_ISI_OUT_FMT_RAW16,
.memplanes = 1,
.colplanes = 1,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
}

After change ".color" parameter to "MXC_ISI_OUT_FMT_RAW16" the output buffer is completely filled with the 10bits RAW from the image sensor:

"B0 0B 80 0B B0 0B 80 0B 80 0B 40 0B 80 0B 90 0A 00 0B 90 0A 00 0B 70 0A 30 0A 60 0A 50 0A 30 0A"

Attached a sample image.

View solution in original post

5 Replies
1,887 Views
gabrielaugustoc
Contributor II

Hi everyone,

I got the solution for this issue:

{
.name = "BA10",
.fourcc = V4L2_PIX_FMT_SGRBG10,
.depth = { 16 },
.color = MXC_ISI_OUT_FMT_RAW16,
.memplanes = 1,
.colplanes = 1,
.mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
}

After change ".color" parameter to "MXC_ISI_OUT_FMT_RAW16" the output buffer is completely filled with the 10bits RAW from the image sensor:

"B0 0B 80 0B B0 0B 80 0B 80 0B 40 0B 80 0B 90 0A 00 0B 90 0A 00 0B 70 0A 30 0A 60 0A 50 0A 30 0A"

Attached a sample image.

View solution in original post

637 Views
mz-fixposition
Contributor II

Hello @gabrielaugustoc 

Thanks for your post, it is very helpful!

Except for adding the RAW format to `mxc_isi_out_formats`, did you also had to add another format to `mxc_isi_src_formats`?

As you previously pointed out, the only formats that are natively in imx8-isi-cap.c are RGB4 and YUV4

/*
 * Pixel link input format
 */
struct mxc_isi_fmt mxc_isi_src_formats[] = {
	{
		.name		= "RGB32",
		.fourcc		= V4L2_PIX_FMT_RGB32,
		.depth		= { 32 },
		.memplanes	= 1,
		.colplanes	= 1,
	}, {
		.name		= "YUV32 (X-Y-U-V)",
		.fourcc		= V4L2_PIX_FMT_YUV32,
		.depth		= { 32 },
		.memplanes	= 1,
		.colplanes	= 1,
	}
};

 
If yes, what did you add here?

I am also interested in perersonhou's question, how did you read the raw buffer?

Best, Moritz

0 Kudos
1,908 Views
petersonhou
Contributor II

hi,gabrielaugustoc

how could you find the frame data from image sensor lost some byte like you said": 00 14 00 34 00 14 00 34 00 3C 00 58 00 68 00 68 00 4C 00 44 00 38 00 38 00 60 00 54 00 34 00 54 00 34 00 24 00 6C 00 24 00 70 00 34 00 68 00 28 00 4C 00 48 00 4C 00 18 00 44 00 3C 00 44 00 54"? from the CSI2RX or ISI or  oscilloscope?

have you try to shift bit of the frame data as the RM mentioned?

0 Kudos
1,928 Views
gabrielaugustoc
Contributor II

Thanks for your response.
Yesterday I was reading the datasheet IMX8DQXPRM.pdf, page 4.925, and I changed the .depth = { 10 } to .depth = { 16 } on the imx8-isi-cap.c, as you have pointed, this change have increased the output buffer to 16bits (10bits RAW and 6bits padding with 0), that is nice .
But unfortunately, the issue still there, I have one byte 0x00 and the other byte with data.
I have attached my changes in the Linux kernel as a patch file.
I am using a Toradex board and I am working on the Toradex branch toradex_4.14-2.3.x-imx (git://git.toradex.com/linux-toradex.git).
Also, I have included here some debug message from the device drivers when I running an application to read images using V4L2:

[   81.125584] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[   87.843953] drivers/media/platform/imx8/mxc-isi-cap.c, mxc_isi_source_fmt_init, 989 CONFIG_VIDEO_ECAM not configured
[   87.862943] drivers/media/platform/imx8/mxc-isi-hw.c, mxc_isi_channel_set_csc, 300 dst_fmt->color = 0x000C
[   87.872730] drivers/media/platform/imx8/mxc-isi-hw.c, mxc_isi_channel_set_csc, 306 is using V4L2_PIX_FMT_SGRBG10
[   87.882993] drivers/media/platform/imx8/mxc-isi-hw.c, mxc_isi_channel_set_csc, 325, bypass csc
[   87.891645] input fmt RGB4
[   87.894394] output fmt BA10
[   87.897192] drivers/media/platform/imx8/mxc-isi-hw.c, mxc_isi_channel_config, 614, CHNL_CTRL_CHNL_BYPASS_ENABLE=1
[   88.258740] drivers/media/platform/imx8/mxc-mipi-csi2.c, mxc_mipi_csi2_hc_config, 240, MIPI Lane = 4
[   88.268003] drivers/media/platform/imx8/mxc-mipi-csi2.c, mxc_mipi_csi2_hc_config, 254, Virtual Channel = 0
[   88.277737] drivers/media/platform/imx8/mxc-mipi-csi2.c, mxc_mipi_csi2_csr_config, 207, CSI2SS_DATA_TYPE = 0xFFF7FFFF
[   88.288388] drivers/media/platform/imx8/mxc-mipi-csi2.c, mxc_mipi_csi2_csr_config, 219, hs_settle = 6
[   89.371574] i.mx8-hdp 56268000.hdmi: No EDID function, use default video mode
[   89.380368] i.mx8-hdp 56268000.hdmi: No EDID function, use default video mode


I am trying to figure out where the "input fmt" was set as RGB4, and if this is the correct for RAW10.
I am thinking the "output fmt" is correct as BA10 "RAW10", but the Pixel Link connection between the MIPI CSI-2 [1] and the ISI is not correct.
I will continue to read the datasheet and device drivers.

My best regards,

Gabriel

0 Kudos
1,939 Views
weidong_sun
NXP TechSupport
NXP TechSupport

Hi gabrielaugustoc ,

Try the setting, please!

In imx8-isi-cap.c, add lines below:

struct mxc_isi_fmt mxc_isi_out_formats[] = {
{
+ .name = "RAW10",
+ .fourcc = V4L2_PIX_FMT_SBGGR10,
+ .depth = { 16 },
+ .color = MXC_ISI_OUT_FMT_RAW10,
+ .memplanes = 1,
+ .colplanes = 1,
+ .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ },

 

B.R,

Weidong