iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

natesigrist — Wed, 16 May 2018 15:27:36 GMT

Does CCM_ANALOG_PLL_VIDEO need to be configured for iMX6DL?
Is the bandwidth (based on 2.3.1 in AN5305):
1920x1080 (pixels/frame) x 10 (bits/pixel) / 8 (bits/cycle) x30 (frames/second) x 1.35 = 105 MHz;

(This post is still being written. I WILL explain how to get a working Bayer conversion at 30fps!)

This is an attempt to capture a complete guide on working with the iMX6 (specific to DL) MIPI CSI2 camera that outputs Bayer format (10-bit). I will attempt to keep it up to date.

I'll start out with our current configuration,

Sensor	OV5675
Pixel Format	Bayer 10-bit
Targeted Resolution	1920x1080
Targeted Frame Rate	30 FPS
MIPI Lanes	2
MIPI Clock (Reference Manual)	900 MHz
Pixel Clock (Reference Manual)	112.5 MHz
VSync	No additional pins / Read from MIPI packet
iMX6DL path	CSI2-->MIPI-->SMFC-->IMDAC-->GPU
MIPI Packet format (reg. 0x4814)	0x2A
Virtual Channel	0
CSI clock source	CLKO => 24MHz
git clone git://git.freescale.com/imx/linux-imx.git -b imx_4.9.11_1.0.0_ga

From 2.3.1 in the AN5305 document describes the payload as being 16-bits wide and 8-bits per cycle. For Bayer 10-bit one clock cycle can send,

10 (bits/pixel) / 8 (bits/cycle) = 5/4 (cycles/pixel)

The Camera drives the MIPI clock and can be calculated,

Bandwidth: (must be < 125 MHz)

1920x1080 (pixels/frame) x 10 (bits/pixel) / 8 (bits/cycle) x30 (frames/second) x 1.35 = 105 MHz

MIPI D-PHY Data Rate:

105 MHz x 8 (bits/cycle) = 840 Mb/s

MIPI Clock:

840 Mb/s / 2 (lanes) / 2 (reads/bit DDR) = 210 MHz < 900 MHz (actual)

MIPI_CSI2_PHY_TST_CTRL1:

900 MHz (0x34)

Turns out I was relying too much on a crappy oscilloscope that read 450 MHz when the actual speed was 900 MHz. Now the 'calibrated' MIPI D-PHY clock can be set according to table 4 in the MIPI-CSI2 Guide. This works out to be 0x34. This assumes the ref_clock (PLL3->VIDEO_27M_CLK_ROOT) is at 27MHz. This can be verified by checking MIPI_CSI_PHY_STATE, (This cannot be verified reliably but may give some clues)

/* Make sure clocks are enabled */

clk_prepare_enable(info->cfg_clk);
clk_prepare_enable(info->dphy_clk);

/* Set lanes to 2 */

mipi_csi2_write(info, info->lanes - 1, MIPI_CSI2_N_LANES);

/* Low is active for the following */

mipi_csi2_write(info, 0x0, MIPI_CSI2_PHY_SHUTDOWNZ);
mipi_csi2_write(info, 0x0, MIPI_CSI2_DPHY_RSTZ);
mipi_csi2_write(info, 0x0, MIPI_CSI2_CSI2_RESETN);

/* Sequence required to 'calibrate' MIPI D-PHY clock */

mipi_csi2_write(info, 0x00000001, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL1);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000002, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00010044, MIPI_CSI2_PHY_TST_CTRL1);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000034, MIPI_CSI2_PHY_TST_CTRL1); /* <-- 900 MHz */
mipi_csi2_write(info, 0x00000002, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL0);

/* Put back high to enable MIPI D-PHY */

mipi_csi2_write(info, 0xffffffff, MIPI_CSI2_PHY_SHUTDOWNZ);
mipi_csi2_write(info, 0xffffffff, MIPI_CSI2_DPHY_RSTZ);
mipi_csi2_write(info, 0xffffffff, MIPI_CSI2_CSI2_RESETN);

/***** Put Camera sensor into MIPI STOPSTATE *****/

/* Wait Camera MIPI lines to be in state LP-11 (STOPSTATE) */
while (i++<10 && ((status = mipi_csi2_dphy_status(mipi_csi2)) & 0x430) != 0x430)

msleep(10);

if (i >= 10) {

dev_err(dev, "Error w/ MIPI CSI2 clock (0x%08x)\n", status);
return -1;

}

/***** Re-enable the Camera sensor here *****/

i = 0;

/* Wait for MIPI stable */
while (i++<10 && ((status1 = mipi_csi2_get_error1(mipi_csi2)) ||

(status2 = mipi_csi2_get_error2(mipi_csi2))))

msleep(10);

/* In case it was missed and to satisfy the compiler. */

status2 = mipi_csi2_get_error2(mipi_csi2);

if (i >= 10) {

dev_err(dev, "mipi csi2 can not reveive data correctly! (0x%08x)\n", status);
return -1;

}

If you still get errors from MIPI_CSI_ERROR1|2 then increase the MIPI_CSI2_PHY_TST_CTRL1 speed. The MIPI D-PHY should have no problems if set above the measured/calculated value. After the camera has been enabled you may notice MIPI_CSI_PHY_STATE changing states between 0x00000330 and 0x00000300 as data is being fed into the MIPI D-PHY. Probing with an oscilloscope should also reveal MIPI packets.

If you do not see any packet being transmitted then there is a good chance that your camera is not configured correctly. The OmniVision cameras are known to not transmit when incorrectly configured. Try configuring the camera for just sending test images. For the OV5675, write 0x80 to register 0x4503.

It is recommended that the camera sensor be placed into the MIPI STOPSTATE. The OV5675 can be set to this state by disabling the software i.e. writing 0x00 to register 0x0100. Optionally, restart the OV5675's software by writing 0x01 to register 0x0103.

I initially decided to configure the CCM_ANALOG_PLL_VIDEO clock since it is mentioned in the MIPI-CSI2 Peripheral document example (link was provided above). The CCM_ANALOG_PLL_VIDEO clock (a.k.a. PLL5 - default 630 MHz) and can be adjusted by (18.5.1.3.4 Audio / Video PLL from IMXSDLRM),

PLL5 output frequency = Fref * (DIV_SELECT + NUM/DENOM)

Since the clock is connected to the display output, I decide NOT to continue down this path.

The next step is configuring the CSI-2/IPU Gasket. I might be wrong here, but according to 38.4.3.6.2 in the Application Processor Reference Manual: non-gated mode should be selected for MIPI. I am uncertain if CSI2IPU_SW_RST[CLK_SEL] and IPU_CSI0_SENS_CONF[CSI0_EXT_VSYNC] are related. According to this post the VSYNC is not required. I know they are not the same thing, however something must be used to determine the EOF (End Of Frame). The EOF and VSYNC should come from the MIPI [short] packets.

Now the pixel clock between the gasket and IPU must be configured. Documentation seems to be limited on exactly how to do this. Here is one confusing line provided by NXP,

hsp_clk > ccm_pixel_clk/0.9 > (mipi_clk_lane frequency /(8bits*2))*Data_lane_number

Now the only reference to ccm_pixel_clk in IMX6SDLRM is in table 18.4 System Clocks under MIPI_CORE. Turns out that the ccm_pixel_clk is ipu_hsp_clock for the iMX6SDL and that the hsp_clk > ccm_pixel_clk/0.9 rule only applies to iMX6DQ. As stated here in question 2.

From table 42-1 in the IMX6SDLRM the pixel_clk root is set to ipu1_ipu_hsp_clk_root. Just checking the Linux clk_summary did not reveal this clocks frequency,

cat /sys/kernel/debug/clk/clk_summary | grep hsp

But you can see the routing for this clock in 18.3 of the IMX6SDLRM which shares with MIPI_PIXEL_CLK... You wont be able to find "pixel" in the clk_summary either. There is an ipu1_pclk[01] entry however. ...Taking a step back,

CSCDR3[IPU1_HSP_CLK_SEL] --> CSCDR3[IPU1_HSP_PODF] --> IPU1_HSP_CLK_ROOT & MIPI_PIXEL_CLK

^--> PFD1(540), MMDC CH0, PFD2(396), or PLL3(480) / 4

Turns out that the these two clocks are,

CSCDR3[0x020C403C] = 0x00014e41

ipu1_hsp_clk_sel = 0x3 (540M PFD)

ipu1_hsp_podf = 0x1 (/2)

==> 540/2 = 270 MHz

This clock can be read from the terminal,

$ cat /sys/kernel/debug/clk/clk_summary | grep -E "ipu1_sel|ipu1_podf"
ipu1_sel 0 1 540000000 0 0
ipu1_podf 0 1 270000000 0 0

So IPU1_HSP_CLK_ROOT and MIPI_PIXEL_CLK are 270MHz which is the maximum speed for these clocks. Now we can confirm the condition mentioned above,

IPU1_HSP_CLK (270 MHz) > 900 MHz / (8 (bits/cycle) x 2 (DDR)) x 2 (Lanes) = 112.5 MHz

Pressing on, the IPU must be configured. The goal is to transfer the camera image directly to memory since the IPU cannot process Bayer/Raw pixel format. This is okay since the GPU can be used to convert to other formats (more on that later). So the goal is to go from CSI2IPU --> SMFC --> IDMAC --> GPU.

First confirm that IOMUXC_GPR1[MIPI_IPU1_MUX] is 0 for "enable MIPI to IPU1 CSI0—the virtual channel is fixed to 0".

Re: iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

weidong_sun — Tue, 05 Jun 2018 03:27:44 GMT

Hello Nate,

i.MX6DL doesn't support ISP function, so it requires camera module supports ISP with the function of image format conversion. I checked OV5675 from OV website, and fount it's output format is RAW, I am not sure if ISP has been added to camera module you are using.

By default, our linux BSP supports OV5642, OV5640 etc, these 2 camera modules' output format are both YUV. I have ever ported OV5645 to I.MX6Q/I.MX6DL, see the link, please!

Porting ov5645 to android jb4.2.2

Below is from OV official website:

Have a nice day!

TIC weidong sun

Re: iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

natesigrist — Tue, 05 Jun 2018 15:21:36 GMT

What is ISP?

I know it is a long post, but I stated above that the camera is Bayer 10-bit and that I intend to use the GPU to do the color conversion. The only course to take here is to get the image data to the GPU as quickly as possible (Camera --> MIPI --> IDMAC --> RAM/GPU). Skip any of the IPU functions as they are not helpful here. This is most likely going to be the trend with this processor since it is difficult to find a camera sensor that outputs in other formats. Besides, Bayer is a common format that should have been supported by the iMX6 processor family.

At present, I have a working board with a camera that outputs YUV422 but is capable of outputing Bayer 10-bit. I will try to get this working before I come back to this post.

Re: iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

natesigrist — Mon, 11 Jun 2018 19:08:49 GMT

Got hit with the dreaded "Assumed Answered" again. What gives NXP?

i.MX Processors中的主题 iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

Re: iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

Re: iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer

Re: iMX6[DL] Troubleshooting MIPI CSI2 and Working with Bayer