i.MX8MN EVK MIPI integration for OmniVision OV9282 RAW10

[jkorinth]

Dear *,

I am working on a prototype using the i.MX8MN EVK with an OmniVision OV9282 camera sensor. The OV9282 is a 10b/8b monochrome sensor with a resolution of 1280x800. I'm using a custom Yocto image that is based on `imx-image-core` and have successfully tested the camera pipeline with the MINISAS camera module featuring the OV5640. However, after a lot of testing and experimentation I still can't get reasonable images from the OV9282.

Set OV9282 to a test pattern (8bar linear gradient white->black).
Output: Noise with a horizontal pattern (see attached image).

Our goal is to get RAW10 output from the camera. After unsuccessful experimentation with the imx8-media-dev formats for RGB, we added Y10 and Y8 formats to `imx8-isi-cap.c` driver and got the attached image.

Output of dmesg:

[ +0.077443] ov9282 2-0060: received sequence from userspace, length = 1
[ +0.006738] isi-capture 32e20000.isi:cap_device: cap_vb2_buffer_prepare
[ +0.006697] isi-capture 32e20000.isi:cap_device: cap_vb2_buffer_prepare
[ +0.006649] isi-capture 32e20000.isi:cap_device: cap_vb2_buffer_prepare
[ +0.006682] isi-capture 32e20000.isi:cap_device: mxc_isi_cap_streamon
[ +0.011514] input fmt RGB4
[ +0.002721] output fmt RG10
[ +0.002828] mxc-isi 32e20000.isi: mxc_isi_channel_set_scaling: no scale
[ +0.006668] isi-capture 32e20000.isi:cap_device: cap_vb2_start_streaming
[ +0.008676] isi-capture 32e20000.isi:cap_device: cap_vb2_start_streaming: num_plane=0 discard_size=2048000 discard_buffer=00000000f946011a
[ +0.455596] isi-capture 32e20000.isi:cap_device: mxc_isi.0.capture is no v4l2 subdev
[ +0.042714] isi-capture 32e20000.isi:cap_device: mxc_isi_cap_streamoff
[ +0.008164] isi-capture 32e20000.isi:cap_device: mxc_isi.0.capture is no v4l2 subdev
[ +0.007818] isi-capture 32e20000.isi:cap_device: cap_vb2_stop_streaming

Current devicetree:

/dts-v1/;

#include "imx8mn-evk.dts"

#define PULLUP_1X 0x140

#define PULLDN_1X 0x100

/ {

model = "trinamiX Emu";

compatible = "trinamix,emu", "fsl,imx8mn-evk", "fsl,imx8mn";

chosen {

stdout-path = &uart2;

};

};

&uart1 { status = "disabled"; };

&uart2 { status = "okay"; };

&uart3 { status = "disabled"; };

&micfil { status = "disabled"; };

&ov5640_mipi_0 {

status = "disabled";

};

&mipi_csi_1 {

#address-cells = <1>;

#size-cells = <0>;

status = "okay";

port@0 {

reg = <0>;

mipi1_sensor_ep: endpoint {

data-lanes = <2>;

remote-endpoint = <&trinamix_mipi>;

//csis-hs-settle = <14>;

//csis-clk-settle = <2>;

//csis-wclk;

};

};

};

&i2c3 {

clock-frequency = <400000>;

i2c-scl-rising-time-ns = <168>;

i2c-scl-falling-time-ns = <4>;

status = "okay";

trinamix_ov9282: trinamix-ov9282@60 {

status = "okay";

reg = <0x60>;

compatible = "trinamix,ov9282";

clocks = <&clk IMX8MN_CLK_CLKO1>;

clock-names = "xvclk";

assigned-clocks = <&clk IMX8MN_CLK_CLKO1>;

assigned-clock-parents = <&clk IMX8MN_CLK_24M>;

assigned-clock-rates = <24000000>;

csi_id = <0>;

mipi_csi;

pinctrl-names = "default", "sleep";

pinctrl-0 =

<&pinctrl_csi_pwn>,

<&pinctrl_csi_rst>,

<&trinamix_gpios>,

<&trinamix_ldos>,

<&trinamix_vsyncs>;

pinctrl-1 =

<&pinctrl_csi_pwn>,

<&pinctrl_csi_rst>,

<&trinamix_gpios>,

<&trinamix_ldos>,

<&trinamix_vsyncs>;

/* LDO enable signals */

ir_ldo_1v2_en-gpios = <&gpio3 20 GPIO_ACTIVE_HIGH>;

ir_ldo_2v8_en-gpios = <&gpio3 23 GPIO_ACTIVE_HIGH>;

ir_rgb_ldo_1v8_en-gpios = <&gpio1 21 GPIO_ACTIVE_HIGH>;

/* LED / Laser driver signals */

led_en-gpios = <&gpio5 26 GPIO_ACTIVE_HIGH>;

laser_drv_fail-gpios = <&gpio5 24 GPIO_ACTIVE_HIGH>;

laser_trigger_en-gpios = <&gpio5 27 GPIO_ACTIVE_HIGH>;

laser_driver_en-gpios = <&gpio3 24 GPIO_ACTIVE_HIGH>;

/* Vsync inputs */

ir_cam_vsync-gpios = <&gpio5 10 GPIO_ACTIVE_HIGH>;

/* Shutdown signals */

cam_shutdown-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;

interrupt-parent = <&gpio5>;

interrupts = <GIC_SPI 10 IRQ_TYPE_EDGE_RISING 0>;

interrupt-names = "gpio1_int";

port {

trinamix_mipi: endpoint {

//data-lanes = <1>;

//clock-lanes = <0>;

remote-endpoint = <&mipi1_sensor_ep>;

link-frequencies = /bits/ 64 <400000000>;

};

};

};

trinamix_lm3644: trinamix-lm3644@63 {

status = "okay";

reg = <0x63>;

compatible = "trinamix,lm3644";

pinctrl-names = "default";

pinctrl-0 = <&trinamix_lm3644_gpios>;

};

trinamix_rtmv20: trinamix-rtmv20@34 {

status = "okay";

reg = <0x34>;

compatible = "trinamix,rtmv20";

pinctrl-names = "default";

pinctrl-0 = <&trinamix_rtmv20_gpios>;

};

};

&iomuxc {

trinamix_gpios: trinamix-gpios {

fsl,pins = <

/* LED_TRIGGER_EN */

MX8MN_IOMUXC_UART3_RXD_GPIO5_IO26 PULLUP_1X

/* LASER_TRIGGER_EN */

MX8MN_IOMUXC_UART3_TXD_GPIO5_IO27 PULLDN_1X

/* LASER_DRV_FAIL */

MX8MN_IOMUXC_ECSPI2_MOSI_GPIO5_IO11 0x0

/* IR_STROBE */

MX8MN_IOMUXC_ECSPI2_MISO_GPIO5_IO12 0x0

>;

};

trinamix_ldos: trinamix-ldos {

fsl,pins = <

/* IR_LDO_1V2_EN */

MX8MN_IOMUXC_SAI5_RXC_GPIO3_IO20 PULLUP_1X

/* IR_LDO_2V8_EN */

MX8MN_IOMUXC_SAI5_RXD2_GPIO3_IO23 PULLUP_1X

/* IR/RGB_LDO_1V8_EN */

MX8MN_IOMUXC_SAI5_RXD0_GPIO3_IO21 PULLUP_1X

>;

};

trinamix_lm3644_gpios: trinamix-lm3644-gpios {

fsl,pins = <

/* LED_DRIVER_EN */

MX8MN_IOMUXC_SAI5_RXD1_GPIO3_IO22 PULLUP_1X

>;

};

trinamix_rtmv20_gpios: trinamix-rtmv20-gpios {

fsl,pins = <

/* LASER_DRIVER_EN */

MX8MN_IOMUXC_SAI5_RXD3_GPIO3_IO24 PULLUP_1X

>;

};

trinamix_vsyncs: trinamix-vsyncs {

fsl,pins = <

/* IR_CAM_VSYNC */

MX8MN_IOMUXC_ECSPI2_SCLK_GPIO5_IO10 0x0

>;

};

};

After reading several forum threads here on nxp.com, I've played around with the values for `csis-hs-settle` and `csis-clk-settle`, but with no effect. Do you have any idea what else I could try?

Thanks a lot!
Jens

[john_phillippe]

Was a solution ever found for this issue?

[adrian_alonso]

Hi @john_phillippe 

Not completely, as software team currently doesn't have a camera sensor that supports RAW10 format.

On NXP Linux release L 5.10.72-2.2.0 imx8-mipi-csi2-sam.c added RAW10 support

https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/staging/media/imx/imx8-mipi-csi2-s...

But drivers/staging/media/imx/imx8-isi-cap.c doesn't include RAW10 support.

Let me check with R&D if they got new sensor as current one used on imx8mn is end of life and there was plan to use a new one with RAW8/10 support.

Regards

Adrian 

[adrian_alonso]

Hi @jkorinth,

From quick inspection seems that RAW8/10 is not supported from ISI driver, let me get more info about current status of driver.

Can you let us know which kernel version or Yocto release you are using?

Thanks

[jkorinth]

Hi @adrian_alonso ,

thanks for your response! I'm using the `imx-image-core` image from imx-5.10.52-2.1.0.xml and a patch to support RAW10 that's based on a similar patch for imx6, that one of your colleagues was able to send us:

---
drivers/staging/media/imx/imx8-isi-cap.c | 8 ++++++++
drivers/staging/media/imx/imx8-mipi-csi2-sam.c | 3 ++-
2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/media/imx/imx8-isi-cap.c b/drivers/staging/media/imx/imx8-isi-cap.c
index 57ec7b4354b1..b8ae97b7c395 100644
--- a/drivers/staging/media/imx/imx8-isi-cap.c
+++ b/drivers/staging/media/imx/imx8-isi-cap.c
@@ -108,6 +108,14 @@ struct mxc_isi_fmt mxc_isi_out_formats[] = {
.memplanes = 1,
.colplanes = 1,
.mbus_code = MEDIA_BUS_FMT_RGB888_1X24,
+ }, {
+ .name = "RAW10 (SGRBG10)",
+ .fourcc = V4L2_PIX_FMT_SGRBG10,
+ .depth = { 16 },
+ .color = MXC_ISI_OUT_FMT_RAW10,
+ .memplanes = 1,
+ .colplanes = 1,
+ .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10,
}
};

diff --git a/drivers/staging/media/imx/imx8-mipi-csi2-sam.c b/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
index 928c64d3be49..9215ead8bd2f 100644
--- a/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
+++ b/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
@@ -989,6 +989,7 @@ static void disp_mix_gasket_config(struct csi_state *state)
if (fmt_val == GASKET_0_CTRL_DATA_TYPE_YUV422_8)
val |= GASKET_0_CTRL_DUAL_COMP_ENABLE;
val |= GASKET_0_CTRL_DATA_TYPE(fmt_val);
+ val |= 1 << 2;
regmap_write(gasket, DISP_MIX_GASKET_0_CTRL, val);

if (WARN_ON(!mf->width || !mf->height))
@@ -1138,7 +1139,7 @@ static int mipi_csis_set_fmt(struct v4l2_subdev *mipi_sd,
}

format->pad = source_pad->index;
- mf->code = MEDIA_BUS_FMT_UYVY8_2X8;
+ mf->code = MEDIA_BUS_FMT_SGRBG10_1X10;
ret = v4l2_subdev_call(sen_sd, pad, set_fmt, NULL, format);
if (ret < 0) {
v4l2_err(&state->sd, "%s, set sensor format fail\n", __func__);

I've tried several more things in the meantime, e.g., using SRGGB10_1X10, Y10_1X10 and some other raw bayer formats; the bit that is set manually in the patch appears to be GASKET_0_CTRL_DUAL_COMP_ENABLE; I'm not sure if it really should be set or not, but I've tried both, with the same results.

I'm also fairly sure that `data-lanes = <1 2>` on both ends is the correct configuration; the partial data I've obtained was the result of misconfiguration: An AXI IRQ error occurs, and the driver does streamoff, which is why the capture finishes at all. With the correct configuration, the capture with `v4l2-ctl` simply hangs. Does respond to Ctrl-C, though.

I've also attached a logic analyzer and can see that there's VSYNCs coming from the sensor at the expected speed of 60Hz, so the sensor seems to be running and configured correctly.

More experiments I did in the meantime: switch to 1-lane MIPI, use different link frequencies from 200-800Mhz, even tried to switch camera and cap to RAW8, none of which worked.