i.MX8 ISI for RAW8 and RAW10 formats (grayscale image sensors)

[mz-fixposition]

Dear NXP support,

I am integrating a monochrome image sensor (OV9281) with an i.MX8M Plus.

The sensor can be configured to output RAW8 or RAW10 formats, the related media bus codes are MEDIA_BUS_FMT_Y8_1X8 or MEDIA_BUS_FMT_Y10_1X10.

Now I am wondering what steps I would have to take in order to configure the ISI and patch the ISI source code (imx8-isi-cap.c) so that it can handle these formats?

If I read the ISI driver correctly, then only RGB32 and YUV32 are supported as source formats. Is that correct?

struct mxc_isi_fmt mxc_isi_src_formats[] = {
{
		.name		= "RGB32",
		.fourcc		= V4L2_PIX_FMT_RGB32,
		.depth		= { 32 },
		.memplanes	= 1,
		.colplanes	= 1,
	}, {
		.name		= "YUV32 (X-Y-U-V)",
		.fourcc		= V4L2_PIX_FMT_YUV32,
		.depth		= { 32 },
		.memplanes	= 1,
		.colplanes	= 1,
	}
};

Also in the output formats I cannot find any raw formats (mxc_isi_out_formats).

Optimally I would like to just bypass the ISI altogether so that I could read GRAY8/RAW8 directly from the v4l2src pad.

For example, like this

gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=1 ! 'video/x-raw, format=(string)GRAY8, width=1280, height=800' ! ... 

What would your suggestion be?

@joanxie  @igorpadykov 

Thank you for your advice

Moritz

[mz-fixposition]

Thanks, @joanxie 

Yes, imx8mp does use the same ISI as as the imx8qxp/imx8qm
Alright, so the solution is to use the ISP and remove the ISI from pad link chain?

What are the steps to create the v4l2 capture devices with the ISP instead of the ISI?

At the moment the imx-media-capture driver always links the pads in this way:

mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
mx8-img-md: created link [ov9281 1-0060] => [mxc-mipi-csi2.0]

[joanxie]

one can refer to the dts file, for example,

"https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mp-evk-b...

one can disable or enable in the dts file

[mz-fixposition]

Thanks for the advice

Involving the ISP seems to be a pretty complex process.
Just modifying the device tree will not be enough. (judging from the discussion in open issues)
- https://community.nxp.com/t5/i-MX-Processors/iMX8MP-ISP-Camera-Porting-Guide-for-MIPI-1-lane-sensor/...
- https://community.nxp.com/t5/i-MX-Processors/IMX8M-Plus-Camera-Integration-and-using-ISP/td-p/126070...

Is there really no way to just pass raw format directly to the v4l2 device, without the ISP?

For RAW8/RAW10 monochrome no processing or de-beyering is needed. Using the ISP seems like an overkill.

Best, Moritz

[mz-fixposition]

I just read that RAW formats are generally not supported by all i.MX8 boards. Is that true?

https://github.com/alliedvision/documentation/blob/master/Toradex_Apalis.md

I saw this post from NXP on the issue:
https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX8QXP-capture-raw-bayer-data-and-deba...

The posts suggests patching the Parallel CSI interface in (imx8-parallel-csi.c).
But if I understand correctly, that interface is not even used on the i.MX8M PLUS, which is using the Samsung MIPI-CSI interface (imx8-mipi-csi2-sam.c).

[joanxie]

imx8mp should use the same ISI IP as imx8qxp, for csi, yes, imx8mp uses samsung

[joanxie]

if customer uses imx8mp to connect raw data sensor, why didn't they use ISP?

[joanxie]

as I known, imx8mp has the same as imx8qxp/imx8qm, when ISI capture the raw data, need to be shifted