MIPI CSI-2 interface on iMX7D

[hbeij]

Dear all,

I'm preparing to integrate a new CMOS sensor on the IMX7SABRESD board. I have studied the reference manual (IMX7DRM rev.01) and the relevant kernel drivers (drivers/media/platform/mxc/subdev/mx6s_capture.c and ./drivers/media/platform/mxc/subdev/mxc_mipi_csi.c). The OV5640 works on this board/kernel. 

The new sensor is CSI2 / DPHY compliant, is monochrome, and outputs RAW10/RAW12 (configurable). I'am experimenting with the mx6s_capture kernel driver to investigate the CSI IP behavior when setting different formats.

Most generally, how to get a CSI2 RAW10 or RAW12 monochrome mode working properly? How to configure the CSI2 and CSI peripherals for RAW12 mode, exactly?

Looking at the datasheet, many things are unclear or simply inconsistent:

1. MIPI-CSI2 connects via internal mux to CSI (Figure 13-1), and CSI outputs to DRAM. MIPI-CSI2 is advertised to support RAW6/7/8/10/12/14 (DRM 13.5.1.1) . However, the DRM mentions that the CSI, which the CSI2 connects to, supports (only?) Bayer(RAW?) 8/10/16 bit. So is it possible to use RAW12 mode? How?
2. CSI_CSICR18 register has MIPI_DATA_FORMAT field (30:25), using the exact same modes as MIPI-CSI2. The DRM says MIPI_DATA_FORMAT is read-only (blanked out - 13.3.7.14). Yet it is writable, and the mx6s_capture writes the mode to this field. Simply a DRM issue? Playing around with the values definitely changes the behavior of the output pixels.
3. The MIPI-CSI2 output signals ("ISP CAM I/F") format (Table 13-19) appear MSB, starting at bit 23, with filled dummy 0's from bits 0. But the CSI periperal input pins/format (Table 13-5, 13-7, 13.3.2) start from bit #0, i.e. ipp_csi_d[0]. For example, in RAW10 mode, CSI2 output bit 23 should internally connect to ipp_csi_d[9]. DRM doesn't tell. See 13.5.3.1.2.2.2 "Normal Data alignment in output Data bus". It looks inverted from the CSI data input bus. 
4. What kind of transformation does CSI do on its input data depending on the CSI_CSICR18.MIPI_DATA_FORMAT field? How can CSI receive "generic data"? 
5. How are CSI_CSICR18.MIPI_DATA_FORMAT and MIPI_CSI2_ISP_CONFIG_CH0.DATAFORMAT related with each other? 

Finally, playing around with the settings in mx6s_capture and the OV5640 sensor:

1. Setting CSI_CSICR18.MIPI_DATA_FORMAT to RAW8 instead of YUV8 and the frame looks OK, except half of it is missing. Setting to RAW10 gives a correct image. Setting to RAW12 gives unrecognizable garbage (i.e. even looking at bit slicing / padding, makes little sense). Setting to RGB888 and enable 24-bit input also scrambles. 
2. Can only set CSI_CSICR18.MIPI_DATA_FORMAT once. The next call to configuration and the DATA_FORMAT reads back 0x3e - which is invalid / non-existant. Only power-cycling works to reset the peripheral. 

Hope someone can answer these questions! 

Regards,

Hendrik

[hbeij]

Update

I have the IMX290 sensor from Sony working on the IMX7.

10bit is supported by the hardware (RAW10 in the drivers).

12 bit is not supported by the hardware. The CSI2 peripheral can receive 12bit data, however, the CSI peripheral cannot correctly understand it, resulting in garbage data.

Source code for the IMX290 will be released shortly (before feb2019) but is available on request.

Thanks @ all for your help and suggestions!

[hbeij]

Hello Ben,

In preparation for the IMX290 driver, I played a bit with OV camera modules and different formats. The SBGGR8_1X8 is working fine. Basically, it just registers 8bpp for the low-level CSI interface. And on another level, it encodes information on how to de-bayer.

No ISP function is not really a problem. It just means you have to control exposure time somehow by software. In practice it limits the framerate, and you might get less responsive/accurate results than with HW ISP.

I have no clue about gstreamer at all. I just use the v4l2grab example program that minimally wraps the V4L2 IOCTLs to get the image buffer directly in a program. Easy to inspect the raw data comming out the camera :-)

I'll work on the driver this and next month. If it works, I will try to mainline the driver code.

KR,

Hendrik

[jhongting]

Dear Hendrik Beijeman,

My method same as you , I'm use via v4l2 ioctl get raw data , bayer to RGB888 via RAW Pixels get the image.

Refer link:
    standard v4l2
    https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/capture.c.html

    http://blog.csdn.net/zgyulongfei/article/details/7526249
    http://blog.csdn.net/LavenderSs/article/details/51495648

According your description maybe need control exposure time by SW.

I have the bpp question , in mx6s_capture.c nxp code bpp fill in 1 . what's csi interface set 8bpp?

, {
        .name        = "RAWRGB8 (SBGGR8)",
        .fourcc        = V4L2_PIX_FMT_SBGGR8,
        .pixelformat    = V4L2_PIX_FMT_SBGGR8,
        .mbus_code    = MEDIA_BUS_FMT_SBGGR8_1X8,
        .bpp        = 1,
    },

whatever I have no idea to test my imx219 sensor.

Do you know (SBGGR10_1X10) bpp is 8 or 10 and any idea suggestion?

Thanks your reply , hope successes IMX290.

Thanks,

Ben

[hbeij]

Hi Igor,

Thanks for the info, I saw these before, but double checked them. According to OV5647 datasheet it outputs RAW8/RAW10. The ov5647_mipi.c driver is just a I2C / V4L wrapper. Apparantly the ov5647_mipi driver does some software ISP functions for exposure time etc.

Pixel format is fixed to MEDIA_BUS_FMT_SBGGR8_1X8, and the CSI driver imx6c_capture translates this to BIT_MIPI_DATA_FORMAT_RAW8 in the CR18 register. 

I would need at least RAW10, preferably RAW12. My sensor also has no ISP, but I don't really need it. Exposure time is fixed and frame rates are low. I will make a custom driver and not use V4L at all (or only if convenient). No need for gstreamer or v4l tools. 

So assuming the OV5647 works in RAW8 mode, that is good news. Has anyone tried RAW10 mode? 

My needs are simple, basically I just need to know if the RAW12 mode will work to get the sensor frame delivered, that's all... (I'm still unsure because CSI2 says its supported but CSI doesn't mention it).

Kind regards!

Hendrik

[jhongting]

Hello Hendrik Beijeman,

May I ask ov5647 pixel format set to MEDIA_BUS_FMT_SBGGR8_1X8 is working?

Because my HW is imx8 same as imx7 has no ISP function , it's not working for me.

I'm used "gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink" test always fail.

is possible share driver code and dtsi?

Thanks,

Ben

[igorpadykov]

Hi Hendrik

in 4.1.15 BSP release, there is ov5647_mipi driver for iMX7: drivers/media/platform/mxc/subdev/ov5647_mipi.c.

Interface OV5647 with IMX-7 on MIPI CSI 

i.MX6 OV5647 Bayer sensor driver (ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0) 

AN5305 MIPI–CSI2 Peripheral on i.MX6 MPUs

https://www.nxp.com/docs/en/application-note/AN5305.pdf 

unfortunately I am not aware of additional documentation for mipi-csi module

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[jhongting]

Dear nxp,

Could you reply Hendrik Beijeman question?

[igorpadykov]

Hi Hendrik, ben

unfortunately I am not aware of any RAW12 cameras tested with i.MX7D.

Please note that NXP has special service for helping customers to port custom drivers:

NXP Professional Services|NXP 

Best regards
igor