topic Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller. in i.MX Processors

mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Thu, 12 Jan 2017 07:12:53 GMT

Hi,

I am trying to interface omnivision's ov9724 camera with imx6dl microcontroller. The camera has only mipi interface, and I am using only one lane for communication (one differential data lane and a differential clock lane).

iMx6dl pin OV9724 pin

CSI0_MCLK(P4) - XCLK -> Source clock: 24 MHz

CSI_CLK0M(F4) - MCN (MIPI_CLK_N)

CSI_CLK0P(F3) - MCP (MIPI_CLK_P)

CSI_D0M(E4) - MDN0 (MIPI_D0_N)

CSI_D0P(E3) - MDP0 (MIPI_D0_P)

I ported the camera driver for ov5640 in "git://git.freescale.com/imx/linux-2.6-imx.git". Since the camera only has RAW-10 bit output, I edited the code based on a reference code I found online.

This is the relevant dts configuration I have edited.

ov9724_mipi: ov9724_mipi@10

{
compatible = "ovti,ov9724_mipi";
reg = <0x10>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_ipu1_3>;
clocks = <&clks IMX6QDL_CLK_CKO>; //&clks 201 ??

clock-names = "csi_mclk";
DOVDD-supply = <&sw4_reg>; /* 1.8v */
AVDD-supply = <&vgen5_reg>; /* 2.8v, on rev C board is VGEN3,
on rev B board is VGEN5 */
DVDD-supply = <&vgen1_reg>; /* 1.5v*/
pwn-gpios = <&gpio6 2 GPIO_ACTIVE_LOW>; /* active low: CSI0_DAT16 - PWRDWN*/ //REF MANUAL PG : 1523
stby-gpios = <&gpio6 0 GPIO_ACTIVE_LOW>; /* active low: CSI0_DAT14 - STANDBY - XSHUTDOWN*/
csi_id = <0>;
mclk = <24000000>;
mclk_source = <0>;
};

&mipi_csi {
status = "okay";
ipu_id = <0>;
csi_id = <0>;
v_channel = <0>; //v_channel 0: CSI0_IPU1; v_channel1: CSI1_IPU1; 2: CSI0_IPU2; 3: CSI1_IPU2
lanes = <1>;
};

v4l2_cap_0 {
compatible = "fsl,imx6q-v4l2-capture";
ipu_id = <0>;
csi_id = <0>;
mclk_source = <0>;
status = "okay";
};

v4l2_out {
compatible = "fsl,mxc_v4l2_output";
status = "okay";
};

I am attaching the error log below. It looks like a clock issue to me, as referring to the errors, I find that these two bits are set according to the datasheet in Error register 1.

Bit 4: Error matching Frame Start with Frame End for Virtual Channel 0

Bit 28: Header ECC contains 2 errors. Unrecoverable.

I have configured the imx to use IPU - 1, CSI - 0, Virtual Channel - 0 and Number of lanes - 1. Please refer to the log below.

I think it might be something to do with the clock configuration but I am unsure how to go forward. I saw the dphy register settings mentioned in the "Debug Steps for customer MIPI Sensor" doc, which I think the author created based on the info available in document AN5305 (Page 14), which I am attaching here.

I configured the clock as follows.

In the camera sensor side according to the datasheet,

Pixel clk = (ext_clk * pll_multiplier) / (sys_clk_div_pll * pre_pll_clk_div_pll * pix_clk_div_pll)

= (24000000 * 0x3E) / (0x0A * 0x01 * 0x02)

= 75.6 MHz.

Now for configuring the mipi dphy clock on the imx side, I used the following calculation. (For 1280 x 720 at 30 fps)

(This calculation is based on the equations in AN5305 doc attached here, Section 3.4, Page 13).

Pixel clock = 1280 * 720 * 30 fps * 1 cycle/pixel * 1.35 blanking interval = 74.6 MHz

Total MIPI Data rate = 74.6 * 10 bits = 746 Mbps.

For a 1 lane interface,

MIPI clock = 746 / (Number of lanes ) / 2 = 746 / 1 / 2 = 373 MHz.

MIPI_CSI2_PHY_TST_CTRL1 setting = 373 MHz * 2 (DDR mode) = 746 MHz

Based on this value, i edited the mipi dphy settings as follows in mxc_mipi_csi2.c by referring to AN5305 page 14.

mipi_csi2_write(info, 0x00000001, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL1);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000002, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00010044, MIPI_CSI2_PHY_TST_CTRL1);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000012, MIPI_CSI2_PHY_TST_CTRL1); //750-800 MHz
mipi_csi2_write(info, 0x00000002, MIPI_CSI2_PHY_TST_CTRL0);
mipi_csi2_write(info, 0x00000000, MIPI_CSI2_PHY_TST_CTRL0);

I am still not sure the clock settings are correct. I am also unclear as to the clock settings for ov5640 mentioned in AN5305 Page 21. (I have attached it as an image here - MIPI CLK setting in AN5305.png). Why are they setting the PLL5 to 596 MHz?

Also, the ipu and mipi configs are as below.

mipi_csi: mipi_csi@021dc000 { /* MIPI-CSI */
compatible = "fsl,imx6q-mipi-csi2";
reg = <0x021dc000 0x4000>;
interrupts = <0 100 0x04>, <0 101 0x04>;
clocks = <&clks IMX6QDL_CLK_HSI_TX>,
<&clks IMX6QDL_CLK_EMI_SEL>,
<&clks IMX6QDL_CLK_VIDEO_27M>;
/* Note: clks 138 is hsi_tx, however, the dphy_c
* hsi_tx and pll_refclk use the same clk gate.
* In current clk driver, open/close clk gate do
* use hsi_tx for a temporary debug purpose.
*/
clock-names = "dphy_clk", "pixel_clk", "cfg_clk";
status = "disabled";
};

ipu1: ipu@02400000 {
#address-cells = <1>;
#size-cells = <0>;
compatible = "fsl,imx6q-ipu";
reg = <0x02400000 0x400000>;
interrupts = <0 6 IRQ_TYPE_LEVEL_HIGH>,
<0 5 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clks IMX6QDL_CLK_IPU1>,
<&clks IMX6QDL_CLK_IPU1_DI0>, <&clks IMX6QDL_CLK_IPU1_DI1>,
<&clks IMX6QDL_CLK_IPU1_DI0_SEL>, <&clks IMX6QDL_CLK_IPU1_DI1_SEL>,
<&clks IMX6QDL_CLK_LDB_DI0>, <&clks IMX6QDL_CLK_LDB_DI1>;
clock-names = "bus",
"di0", "di1",
"di0_sel", "di1_sel",
"ldb_di0", "ldb_di1";
resets = <&src 2>;
bypass_reset = <0>;
};

I printed the "dphy_clk", "pixel_clk", and "cfg_clk".

sh-4.3# dmesg | grep clk

[ 0.259781] imx-ipuv3 2400000.ipu: ipu_clk = 270000000
MIPI CSI2 cfg_clk: 27000000
MIPI CSI2 dphy_clk: 198000000
MIPI CSI2 pixel_clk: 396000000
[ 0.381171] imx-ipuv3 2400000.ipu: pixel clk = 30919000
[ 0.381240] imx-ipuv3 2400000.ipu: try ipu internal clk
[ 0.381253] imx-ipuv3 2400000.ipu: rounded pix clk:30000000
[ 0.381258] imx-ipuv3 2400000.ipu: try ipu ext di clk
[ 0.381477] #### clk_pllv3_av_set_rate : rate 989407992, parent_rate 24000000, val 0x0, mfn 0x37035 mfd 0xf4240
[ 0.381509] imx-ipuv3 2400000.ipu: di clk:30919000
[ 0.381525] imx-ipuv3 2400000.ipu: round pixel clk:30919000
[ 0.428329] imx-ipuv3 2400000.ipu: pixel clk = 30919000
[ 0.428387] imx-ipuv3 2400000.ipu: try ipu internal clk
[ 0.428402] imx-ipuv3 2400000.ipu: rounded pix clk:30000000
[ 0.428408] imx-ipuv3 2400000.ipu: try ipu ext di clk
[ 0.428429] imx-ipuv3 2400000.ipu: di clk:30919000
[ 0.428442] imx-ipuv3 2400000.ipu: round pixel clk:30919000
OV9724 Clock csi_mclk: 24000000
[ 2.879715] galcore: clk_get vg clock failed, disable vg!

I saw that the pixel clk has been rounded to 30919000. I did not understand this part. Here the dphy_clk is 198 MHz. Do I have to change any extra pll settings or something?

LOG

# ioctl_g_chip_ident #sensor chip is ov9724_mipi_camera

sensor supported frame size:

In mxc_v4l2_s_param
640x480
320x240
720x480
720x576
# ioctl_g_parm #
1280x720
1920x1080
2592x1944
176x144
1024x768

Current capabilities are 1001
sensor frame format: BG10

Current capturemode is 0 change to 0
sensor frame format: BG10

Current framerate is 30 change to 30
sensor frame format: BG10
sensor frame format:BG10
# ioctl_s_parm #
sensor frame format: BG10
sensor frame format: BG10
sensor frame format: BG10
sensor frame format: BG10
sensor frame format: BG10

# INIT MODE mode: 0 frame rate: 1 mode_original: 0 #

MIPI CSI2 Enable Status: 1
MIPI CSI2 Enable Status: 1
MIPI CSI2 Befor setting Lanes: info->lanes: 1
MIPI CSI2 Set Lanes: 0
MIPI CSI2 Set Datatype : 43 0x2b --> RAW-10 datatype
Pixel Format is V4L2_PIX_FMT_SBGGR10

************ Changing to direct mode! Frame rate is : 1 Mode number is 0

Writing 0x24001b30 to register CSI_SENS_CONF Read Val: 0x4001b30

Writing 0x2cf04ff to register CSI_SENS_FRM_SIZE Read Val: 0x2cf04ff

Writing 0x2cf04ff to register CSI_ACT_FRM_SIZE Read Val: 0x2cf04ff

Writing 0xffffff2b to register IPU_CSI0_DI Read Val: 0xffffff2b

Writing 0x661 to register IPU_CONF Read Val: 0x661

Writing 0x2 to register CSI2IPU_SW_RST Read Val: 0x2

# OV9724 CHANGE MODE DIRECT # Frame Rate: 1 Mode : 4

@@@@@@@@@@@@@@@@@@@ STREAM OFF @@@@@@@@@@@@@@@@@@@@@@@
############## OV9724 REGISTER VALUES READBACK##############
IPU_CONF = 0x661
IPU_CSI0_SENS_CONF_REG = 0x4001b30
IPU_CSI0_SENS_FRM_SIZE_REG = 0x2cf04ff
IPU_CSI0_ACT_FRM_SIZE_REG = 0x2cf04ff
IPU_CSI0_OUT_FRM_CTRL_REG = 0x0
IPU_CSI0_DI_REG = 0xffffff2b
IOMUXC_GPR1_REG = 0x48441005
CSI2IPU_SW_RST_REG = 0x2

@@@@@@@@@@@@@@@@@@@ STREAM ON @@@@@@@@@@@@@@@@@@@@@@@
Setting Virtual Channel to 0 Channel Reg Value: 2b
GEtting CSI Ready!!
############## OV9724 REGISTER VALUES ##############
IPU_CONF = 0x661
IPU_CSI0_SENS_CONF_REG = 0x4001b30
IPU_CSI0_SENS_FRM_SIZE_REG = 0x2cf04ff
IPU_CSI0_ACT_FRM_SIZE_REG = 0x2cf04ff
IPU_CSI0_OUT_FRM_CTRL_REG = 0x0
IPU_CSI0_DI_REG = 0xffffff2b
IOMUXC_GPR1_REG = 0x48441005
CSI2IPU_SW_RST_REG = 0x2

MIPI CSI2 PHY_STATE : 0x300
MIPI_CSI2_VERSION : 0x3130302a
MIPI_CSI2_N_LANES : 0x0
MIPI_CSI2_PHY_SHUTDOWNZ : 0x1
MIPI_CSI2_DPHY_RSTZ : 0x1
MIPI_CSI2_DATA_IDS_1 : 0x0
MIPI_CSI2_DATA_IDS_2 : 0x0
MIPI_CSI2_PHY_TST_CTRL0 : 0x0
MIPI_CSI2_PHY_TST_CTRL1 : 0x2a2a

MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000000
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000000
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000000
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000010
MIPI CSI2 ERROR2 : 0x0
MIPI CSI2 ERROR1 : 0x10000000
MIPI CSI2 ERROR2 : 0x0
mipi csi2 can not receive data correctly!

I am unable to probe the clk and data lines as the DSO I have is only of 200 MHz. The waveforms I did observe I have attached below. (Yellow probe - DATA_P, Green probe - DATA_N, Blue - CLK_P, Pink - CLK_N).

I am observing data on the data lines when turning on the streaming, but did not see anything on the clock line initially. Then when I decreased voltage division of clock lines to 100mV (data voltage is around 1 V), then I saw a waveform pattern in the clock, which looked like noise initially. Seems like clock is present when the data is present. But when I probe both the data lines, clock lines look like noise as shown in fig scope_15.bmp. I do not know why the clock is behaving as such - HARDWARE ISSUE? But in the imx side, a ddr clock is detected as per the MPHY MIPI CSI2 PHY_STATE register.

I know this is a long post, but I wanted to include everything I have done till date. I am a newbie to linux, and basically this is my first project. I will summarize my questions below.

1) What could be the reason for the MIPI CSI error seen in the log?

2) Does the clock in the waveforms look normal? Is it adequate for the mipi interface to work?

3) In case my dphy clock configuration is wrong, how to configure both imx and the camera clock? More importantly, how is the camera sensor clock related to the dphy clock?

4) In the document AN5605 page 21, what is the significance of the Clock 569MHz and what is the clock I should use?

Please let me know your thoughts.

stathisv‌ SaurabhPatel‌ wangjia‌ igorpadykov‌ mko‌ jamesbone‌ gary_bisson‌ wangshengjiu‌ ShaojunWang‌ GaoJianzhong‌ YuriMuhin_ng‌ wallyyeh‌ - I saw that you guys have already worked with imx and omnivision cameras.. It would be helpful if you could take a look and let me know if am I doing anything wrong..

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

wallyyeh — Fri, 13 Jan 2017 02:35:50 GMT

Hi, Navina:

I'm not en expert on this, but I take a look at ADV7480 instruction (that's my only device use MIPI on imx6); 1-Lane seems support max resolution to 800x600; 1280x720 must use 2-Lane for data transfer, could you also check that?

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Fri, 13 Jan 2017 06:03:52 GMT

Hi Wally,

Thanks for your prompt reply.

I went through the ADV7480 spec sheet and saw that it has 4 lanes. The ov9724 has only one single MIPI Lane, and the maximum supported resolution is 1280x720 at 30fps. It can use 1-Lane for data transfer. Is there any info you can share regarding how you went about configuring the dphy clock lanes on the imx side and the clock on camera side?

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

wallyyeh — Fri, 13 Jan 2017 08:08:40 GMT

Hi, Navinar:

I'll take a look while I off the work and see what can I help.

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Fri, 13 Jan 2017 09:49:18 GMT

Thanks a lot wallyyeh‌.

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

shaojun_wang — Fri, 13 Jan 2017 09:55:26 GMT

Please take care of the gpr13 register, the default setting is "IPU1 CSI1 connects to MIPI CSI2 virtual channel 1"

static void __init imx6q_csi_mux_init(void)
{
/*
* MX6Q SabreSD board:
* IPU1 CSI0 connects to parallel interface.
* Set GPR1 bit 19 to 0x1.
*
* MX6DL SabreSD board:
* IPU1 CSI0 connects to parallel interface.
* Set GPR13 bit 0-2 to 0x4.
* IPU1 CSI1 connects to MIPI CSI2 virtual channel 1.
* Set GPR13 bit 3-5 to 0x1.
*/
struct regmap *gpr;

gpr = syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");
if (!IS_ERR(gpr)) {
  if (of_machine_is_compatible("fsl,imx6q-sabresd") ||
   of_machine_is_compatible("fsl,imx6q-sabreauto"))
   regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 19, 1 << 19);
  else if (of_machine_is_compatible("fsl,imx6dl-sabresd") ||
    of_machine_is_compatible("fsl,imx6dl-sabreauto"))
   regmap_update_bits(gpr, IOMUXC_GPR13, 0x3F, 0x0C);
} else {
  pr_err("%s(): failed to find fsl,imx6q-iomux-gpr regmap\n",
         __func__);
}
}

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Fri, 13 Jan 2017 13:30:27 GMT

Hi wangshengjiu‌,

Thanks for your prompt reply. I did verify both the registers, and they look fine.

OV9724 IOMUXC_GPR1_REG = 0x48441005
OV9724 IOMUXC_GPR13_REG = 0xc

In GPR1 reg, bits 19 and 20 are 0 (0x48441005). According to the datasheet, for imx6dl,

Bits 20–19 : 0 Gasket is selected

1 IOMUX is selected

In my case, the CSI2IPU Gasket should be enabled right? So 19-20 should be 0.

And GPR13 register is 0x0c : Bits 3-5: 0x01 and bits 0-2: 100

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

wallyyeh — Sat, 14 Jan 2017 08:07:07 GMT

I believe you already read my posts:

A Simple tutor for writing i.MX6 mipi driver, use adv7480 as an example

I didn't do any thing special when I initialize my ADV7480, but I did mention this:

adv7480_init_mode(), I found after mipi_csi2_reset_with_dphy_freq() or mipi_csi2_reset() called, you *must* turn off device's output and turn it on again! or you will got mipi_csi2_dphy_status alwaly be 0x200 or 0x230, it means i.MX6 can't get the clock form device.

In my ADV7480 driver(which is inside patch 0004 in adv7480_mipi_driver_patches.zip), you can see I write a fancy function named "mipi_csi2_reset_with_dphy_freq()" and use it between adv7480_stream_on() and adv7480_stream_off() in adv7480_change_mode_direct(); this is what you must care I think.

In your previous post, you said MIPI CSI2 PHY_STATE : 0x300; this make me believe your dphy clock settings must be OK, It's normal that it's value shows 0x300 or 0x320. It will continue change between them. I wonder you may need mipi_csi2_reset_with_dphy_freq() during you streamoff/on the OV7942.

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

wallyyeh — Sat, 14 Jan 2017 08:08:35 GMT

I also forget to ask what branch is your kernel, I use 3.0.35; and maybe it's helpful to know your git branch.

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

igorpadykov — Sun, 15 Jan 2017 23:37:43 GMT

Hi Navinar

seems there is error in calculation:

"(This calculation is based on the equations in AN5305 doc attached here, Section 3.4, Page 13).

Pixel clock = 1280 * 720 * 30 fps * 1 cycle/pixel * 1.35 blanking interval = 74.6 MHz"

1280 * 720 * 30 fps * 1 cycle/pixel * 1.35 blanking interval = 37.3 MHz

Best regards
igor

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Mon, 16 Jan 2017 06:22:49 GMT

Hi igorpadykov‌,

Sorry, my mistake. The cycle/pixel value is 2 which is how I obtained 74.6MHz. The raw rgb data is 10 bit, so 2 cycles will be required to send a pixel right?

1280 * 720 * 30 fps * 2 cycle/pixel * 1.35 blanking interval = 74.6 MHz

Best Regards,

Navinar

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

igorpadykov — Mon, 16 Jan 2017 06:31:08 GMT

in such case seems it exceeds 1-lane max. throughput (AN5305, p.9)

62.5 MHz for a 1-lane configuration (1000 Mb/s/lane, 125 MB/s)

Best regards
igor

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Mon, 16 Jan 2017 06:31:08 GMT

Hi wallyyeh‌,

Sorry for the delay in answering. I actually set the PHY_STATE registers before turning on the camera. On initial startup, the camera is off and not in streaming state. I start the streaming only after the dphy configuration is correct, which must be why I am getting the normal value as 0x300.

But there is some good news. I managed to rectify the clock errors by changing some reg configurations. Still errors are present like ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0 but I am hopeful. I am not yet sure if the data is being received correctly. I am attaching a log below, in my main thread, with the latest log. I will also attach some exhaustive logs with gst-debug level 4 output.

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Mon, 16 Jan 2017 06:36:08 GMT

Hi,

I have been able to remove the clock error of mipi by making some changes. I am getting the ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0 still, and not getting any data at all. I am attaching the latest log here.

sh-4.3# ./unit_tests/mxc_v4l2_capture.out -iw 1280 -ih 720 -ow 1280 -oh 720 -m 0 -i 1 -r 0 -c 1 -fr 30 -d /dev/video1 /mnt/test2.yuv

in_width = 1280, in_height = 720
In MVC: mxc_v4l_open

out_width = 1280, out_height = device name is Mxc Camera
720
top = 0, left = 0

# OV9724 ioctl_g_ifparm
# OV9724 ioctl_g_fmt_cap fmt.pix = 1280#
End of mxc_v4l_open: v2f pix widthxheight 1280 x 720
End of mxc_v4l_open: crop_bounds widthxheight 1280 x 720
End of mxc_v4l_open: crop_defrect widthxheight 1280 x 720
End of mxc_v4l_open: crop_current widthxheight 1280 x 720
On Open: Input to ipu size is 1280 x 720

#IPU Set Window Size CSI_ACT_FRM_SIZE Width: 1280, Height: 720
#IPU ipu_csi_init_interface pix_fmt: 0x30314742 //The BG10 format for raw rgb-10

//Printing some sample file formats below.

IPU_PIX_FMT_YUYV : 0x56595559
IPU_PIX_FMT_UYVY : 0x59565955
IPU_PIX_FMT_RGB24 : 0x33424752
IPU_PIX_FMT_BGR24 : 0x33524742
IPU_PIX_FMT_GENERIC : 0x30555049 //This is the IPU format corresponding to RAW-10 format 0x30314742.
IPU_PIX_FMT_GENERIC_16 : 0x32555049
IPU_PIX_FMT_GENERIC_32 : 0x31555049
IPU_PIX_FMT_LVDS666 fourcc : 0x3644564c
IPU_PIX_FMT_LVDS888 : 0x3844564c

#IPU 0x30314742 for SBGGR10
#IPU CSI_SENS_CONF Readback before write: 0x4001b10
#IPU CSI_SENS_CONF Readback after write: 0xb00

#IPU IPU_CSI_CLK_MODE_NONGATED_CLK
imx-ipuv3 2400000.ipu: CSI_SENS_CONF = 0x04001B10
imx-ipuv3 2400000.ipu: CSI_ACT_FRM_SIZE = 0x02CF04FF0x30314742

#OV9724 ioctl_s_power #
Enabling Ov9724 Regulators
### OV9724 ioctl_dev_init ###

# OV9724 INIT MODE mode: 255 frame rate: 1 mode_original: 255 #

MIPI CSI2 Enable Status: 1
MIPI CSI2 Enable Status: 1
MIPI CSI2 Befor setting Lanes: info->lanes: 1
MIPI CSI2 Set Lanes: 0
MIPI CSI2 POWERING ON!!
MIPI CSI2 Set Datatype : 43 0x2b
Pixel Format is V4L2_PIX_FMT_SBGGR10

Ov9724 Mode init 720_1280

OV9724 Setting Virtual Channel to 0 Channel Reg Value: 2b
GEtting CSI Ready!!
############## OV9724 REGISTER VALUES ##############
OV9724 IPU_CONF = 0x10000660
OV9724 IPU_CSI0_SENS_CONF_REG = 0x4001b10
OV9724 IPU_CSI0_SENS_FRM_SIZE_REG = 0x2cf04ff
OV9724 IPU_CSI0_ACT_FRM_SIZE_REG = 0x2cf04ff
OV9724 IPU_CSI0_OUT_FRM_CTRL_REG = 0x0
OV9724 IPU_CSI0_DI_REG = 0xffffff2b
OV9724 IOMUXC_GPR1_REG = 0x48441005
OV9724 IOMUXC_GPR13_REG = 0xc
OV9724 CSI2IPU_SW_RST_REG = 0x0

############## OV9724 REGISTER VALUES ##############
OV9724 IPU_CONF = 0x10000660
OV9724 IPU_CSI0_SENS_CONF_REG = 0x4001b10
OV9724 IPU_CSI0_SENS_FRM_SIZE_REG = 0x2cf04ff
OV9724 IPU_CSI0_ACT_FRM_SIZE_REG = 0x2cf04ff
OV9724 IPU_CSI0_OUT_FRM_CTRL_REG = 0x0
OV9724 IPU_CSI0_DI_REG = 0xffffff2b
OV9724 IOMUXC_GPR1_REG = 0x48441005
OV9724 IOMUXC_GPR13_REG = 0xc
OV9724 CSI2IPU_SW_RST_REG = 0x0

In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c02c5651

# OV9724 ioctl_g_chip_ident #sensor chip is ov9724_mipi_camera

sensor supported frame size:
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c02c564a
640x480In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
320x240In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
720x480In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
720x576In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
1280x720In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
1920x1080In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
2592x1944In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
176x144In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
1024x768In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c564a
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0405602
sensor frame format: BG10In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0405602
sensor frame format: BG10In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0405602
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0cc5616
case VIDIOC_S_PARM

In mxc_v4l2_s_param

# OV9724 ioctl_g_parm #

Current capabilities are 1001

Current capturemode is 0 change to 0

Current framerate is 30 change to 30

# OV9724 ioctl_s_parm #
# OV9724 INIT MODE mode: 0 frame rate: 1 mode_original: 0 #

MIPI CSI2 Enable Status: 1
MIPI CSI2 Enable Status: 1
MIPI CSI2 Befor setting Lanes: info->lanes: 1
MIPI CSI2 Set Lanes: 0
MIPI CSI2 Set Datatype : 43 0x2b
Pixel Format is V4L2_PIX_FMT_SBGGR10

************ Changing to direct mode! Frame rate is : 1 Mode number is 0
# OV9724 CHANGE MODE DIRECT # Frame Rate: 1 Mode : 4
@@@@@@@@@@@@@@@@@@@ STREAM OFF @@@@@@@@@@@@@@@@@@@@@@@
############## OV9724 REGISTER VALUES ##############
OV9724 IPU_CONF = 0x10000660
OV9724 IPU_CSI0_SENS_CONF_REG = 0x4001b10
OV9724 IPU_CSI0_SENS_FRM_SIZE_REG = 0x2cf04ff
OV9724 IPU_CSI0_ACT_FRM_SIZE_REG = 0x2cf04ff
OV9724 IPU_CSI0_OUT_FRM_CTRL_REG = 0x0
OV9724 IPU_CSI0_DI_REG = 0xffffff2b
OV9724 IOMUXC_GPR1_REG = 0x48441005
OV9724 IOMUXC_GPR13_REG = 0xc
OV9724 CSI2IPU_SW_RST_REG = 0x0

@@@@@@@@@@@@@@@@@@@ STREAM ON @@@@@@@@@@@@@@@@@@@@@@@
OV9724 Setting Virtual Channel to 0 Channel Reg Value: 2b
GEtting CSI Ready!!

# OV9724 ioctl_g_ifparm

clock_curr=mclk=24000000

# OV9724 ioctl_g_fmt_cap fmt.pix = 1280#

g_fmt_cap returns widthxheight of input as 1280 x 720

Retrieved format of 0x30314742 from sensor

SETTING FRAME SIZE TO 1280 x 720!!!!!!!!!!!!!!!!!

#IPU Set Window Size CSI_ACT_FRM_SIZE Width: 1280, Height: 720
#IPU ipu_csi_init_interface pix_fmt: 0x30314742

#IPU 0x30314742 for SBGGR10
#IPU CSI_SENS_CONF Readback before write: 0x4001b10
#IPU CSI_SENS_CONF Readback after write: 0x8b00

#IPU IPU_CSI_CLK_MODE_NONGATED_CLK
imx-ipuv3 2400000.ipu: CSI_SENS_CONF = 0x04001B10
imx-ipuv3 2400000.ipu: CSI_ACT_FRM_SIZE = 0x02CF04FF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0045627
case VIDIOC_S_INPUT
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c014563b
case VIDIOC_G_CROP
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl 4014563c
case VIDIOC_S_CROP
Cropping Input to ipu size 1280 x 720

#IPU Set Window Size CSI_ACT_FRM_SIZE Width: 1280, Height: 720
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c00c56c3
case VIDIOC_S_DEST_CROP
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0cc5605
case VIDIOC_S_FMT
In MVC: mxc_v4l2_s_fmt
type=V4L2_BUF_TYPE_VIDEO_CAPTURE
End of mxc_v4l2_s_fmt: v2f pix widthxheight 1280 x 720
End of mxc_v4l2_s_fmt: crop_bounds widthxheight 1280 x 720
End of mxc_v4l2_s_fmt: crop_defrect widthxheight 1280 x 720
End of mxc_v4l2_s_fmt: crop_current widthxheight 1280 x 720
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c008561c
case VIDIOC_S_CTRL
In MVC:mxc_v4l2_s_ctrl
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0145608
case VIDIOC_REQBUFS
In MVC:mxc_streamoff
MVC: In mxc_free_frame_buf
In MVC:mxc_allocate_frame_buf - size=1382400
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0cc5604
case VIDIOC_G_FMT
In MVC: mxc_v4l2_g_fmt type=1
type is V4L2_BUF_TYPE_VIDEO_CAPTURE
End of mxc_v4l2_g_fmt: v2f pix widthxheight 1280 x 720
End of mxc_v4l2_g_fmt: crop_bounds widthxheight 1280 x 720
End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1280 x 720
End of mxc_v4l2_g_fmt: crop_current widthxheight 1280 x 720
Width = 1280 Height = 720 ImaIn MVC:mxc_v4l_ioctl
ge size = 1382400
pixelformat: YIn MVC: mxc_v4l_do_ioctl c0445609
U12
case VIDIOC_QUERYBUF
In MVC:mxc_v4l2_buffer_status
In MVC:mxc_mmap
pgoff=0x2e600, start=0xb6c5b000, end=0xb6dad000
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0445609
case VIDIOC_QUERYBUF
In MVC:mxc_v4l2_buffer_status
In MVC:mxc_mmap
pgoff=0x2e800, start=0xb6b09000, end=0xb6c5b000
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0445609
case VIDIOC_QUERYBUF
In MVC:mxc_v4l2_buffer_status
In MVC:mxc_mmap
pgoff=0x2ec00, start=0xb69b7000, end=0xb6b09000
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl 40045612
case VIDIOC_STREAMON

In MVC:mxc_streamon with pixel format = 0x32315559

In csi_enc_setup with format = 0x32315559

Updated pixel format in csi_enc_setup = 0x30314742

#IPU ipu_csi_get_sensor_protocol

MIPI CSI2 Enable Status: 1
MIPI CSI2 BIND IPU: 0 0x0
MIPI CSI2 BIND CSI: 0 0x0
Get MIPI CSI2 Virtual Channel. Channel Num: 0
MIPI CSI2 Get Datatype : 43 0x2b
MIPI CSI2 PIXCLK???
imx-ipuv3 2400000.ipu: init channel = 15
init channel = 15

MIPI CSI ipu_conf reg: 10000660
#IPU ipu_smfc_init

#IPU _ipu_csi_set_mipi_di num: 0 di_val: 43 csi: 0

#IPU ipu_csi_init channel: 268435392 csi: 0

#IPU CSI_DATA_DEST_IDMAC
imx-ipuv3 2400000.ipu: CSI_SENS_CONF 2 = 0x04001B10
imx-ipuv3 2400000.ipu: CSI_ACT_FRM_SIZE 2 = 0x02CF04FF
imx-ipuv3 2400000.ipu: IPU_CONF = 0x10000660
imx-ipuv3 2400000.ipu: IDMAC_CONF = 0x0000002F
imx-ipuv3 2400000.ipu: IDMAC_CHA_EN1 = 0x00800000
imx-ipuv3 2400000.ipu: IDMAC_CHA_EN2 = 0x00000000
imx-ipuv3 2400000.ipu: IDMAC_CHA_PRI1 = 0x18800003
imx-ipuv3 2400000.ipu: IDMAC_CHA_PRI2 = 0x00000000
imx-ipuv3 2400000.ipu: IDMAC_BAND_EN1 = 0x00000000
imx-ipuv3 2400000.ipu: IDMAC_BAND_EN2 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_CHA_DB_MODE_SEL0 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_CHA_DB_MODE_SEL1 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_CHA_TRB_MODE_SEL0 = 0x00800000
imx-ipuv3 2400000.ipu: IPU_CHA_TRB_MODE_SEL1 = 0x00000000
imx-ipuv3 2400000.ipu: DMFC_WR_CHAN = 0x00000090
imx-ipuv3 2400000.ipu: DMFC_WR_CHAN_DEF = 0x202020F6
imx-ipuv3 2400000.ipu: DMFC_DP_CHAN = 0x00009694
imx-ipuv3 2400000.ipu: DMFC_DP_CHAN_DEF = 0x2020F6F6
imx-ipuv3 2400000.ipu: DMFC_IC_CTRL = 0x00000002
imx-ipuv3 2400000.ipu: IPU_FS_PROC_FLOW1 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_FS_PROC_FLOW2 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_FS_PROC_FLOW3 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_FS_DISP_FLOW1 = 0x00000000
imx-ipuv3 2400000.ipu: IPU_VDIC_VDI_FSIZE = 0x00000000
imx-ipuv3 2400000.ipu: IPU_VDIC_VDI_C = 0x00000000
imx-ipuv3 2400000.ipu: IPU_IC_CONF = 0x40000000
setting up ipu from csi_enc_setup with pixel format = 0x30555049
IPU_PIX_FMT_YUYV : 0x56595559
IPU_PIX_FMT_UYVY : 0x59565955
IPU_PIX_FMT_RGB24 : 0x33424752
IPU_PIX_FMT_BGR24 : 0x33524742
IPU_PIX_FMT_GENERIC : 0x30555049
IPU_PIX_FMT_GENERIC_16 : 0x32555049
IPU_PIX_FMT_GENERIC_32 : 0x31555049
IPU_PIX_FMT_LVDS666 fourcc : 0x3644564c
IPU_PIX_FMT_LVDS888 : 0x3844564c
#IPU ipu_init_channel_buffer in ipu_common.c
pixel_fmt: 0x30555049, width = 1280, height = 720, bytesperline: 1280, u = 0, v = 0
setting IDMAC channel parameters for Generic mode
Detected SMFC Channel
!!! setting RWS enable for generic sensorEnabling IPU channel : fffffc0 Channel_Id: 15
eba 2e600000
eba 2e800000

IPU Enabling CSI : 0, IPU_CONF: 10000760
Updated IPU_CONF: 10000761
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0445611
case VIDIOC_DQBUF
In MVC:mxc_v4l_dqueue
ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0
VIDIOC_DQBUF failed.
buf.index 0
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl 40045613
case VIDIOC_STREAMOFF
In MVC:mxc_streamoff

IPU Disabling CSI!!
imx-ipuv3 2400000.ipu: CSI stop timeout - 5 * 10ms

OV9724 Disabling IPU Channel : 0xfffffc0 Channel_ID: 15
In MVC:mxc_free_frames
In MVC:mxc_v4l_close
In MVC:mxc_streamoff

#OV9724 ioctl_s_power #
Disabling Ov9724 Regulators
mxc_v4l_close: release resource
MVC: In mxc_free_frame_buf
In MVC:mxc_free_frames
sh-4.3#

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Mon, 16 Jan 2017 07:26:28 GMT

Hi,

The ov9724 sensor outputs 10-bit raw data, which the ipu interprets as Generic data only. In the Reference Manual we saw that the cycles/pixel for MIPI CSI2 interface for Generic data is 2 bytes/pixel.

We are using the same 2 bytes/pixel in my calculations. Is it the same as cycles/pixel? Also, ov9724 camera has only one MIPI data lane, and since it supports 720p image capture at 30fps, I assume that one lane is enough. So is there any changes I should make to the calculation below?

1280 * 720 * 30 fps * 2 cycle/pixel * 1.35 blanking interval = 74.6 MHz

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Tue, 17 Jan 2017 10:02:05 GMT

Hi All,

The ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0 error I mentioned was because the parallel port was hardcoded in the kernel_imx/arch/arm/mach-imx/mach-imx6q.c file.

If you want to use MIPI-CSI module, you should set the bits 0-2 of the IOMUXC_GPR13 register as

000 (virtual channel 0) --> This is what I need!

001 (virtual channel 1); 010 (virtual channel 2); 011 (virtual channel 3); 100 (Parallel Port)

The bit 0-2 was set to 100 which was why I was not getting any data. I modified the bits 0-2 as 000 and 3-5 as 000, and voila, I am getting data now.

regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 19, 1 << 19);

regmap_update_bits(gpr, IOMUXC_GPR13, 0x3F, 0x00); --> Changed here from 0x0C to 0x00.

Now I am able to save frames but there are a couple of issues. Since the IPU is outputting generic data, how can i convert the image to a rgb or yuv format? I am also not sure how to decode the data, and am also not sure if the data I receive is in the correct format. The problem is I am unable to output the test pattern, as the TM (Test Mode) pin of the camera is left open- It needs to be pulled up with a pulldown resistor provided for the camera to send a test pattern. Hence I cannot verify if any data loss has occured, nor if any padding is present or not. I am including some lines of data from the output image here taken at 1280x720 resolution, 30fps.

Q: Does it look correct? I noticed that the 3rd and 4th lines (1line = 16bytes) are the same and it recurs consecutively. Is it expected?

I am also including the files I captured - one file containing only one frame, the other containing 10 frames. How do I make sense of the data in the files? Does the frames look like valid raw-10 frames?

nav_10frame.data - Google Drive

nav_1frame.data - Google Drive

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Tue, 17 Jan 2017 10:12:51 GMT

Hi wallyyeh‌,

Sorry, I did not see this question earlier. My imx kernel version is Linux (none) 4.1.15-226061-gb314f0c-dirty.

I have posted some updates below. Could you look through it and let me know if it makes sense?

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Tue, 17 Jan 2017 10:14:20 GMT

Hi S.j. Wang,

In the GPR13 register, bits 0-2 should be 000 for selecting the virtual channel 0 of MIPI CSI0. If it is 100, that means parallel interface is selected. When I set it to 000 as I am using MIPI interface, virtual channel 0, I started receiving data being streamed through the mipi lines. Could you take a look at my latest question?

Thanks,

Navinar

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

wallyyeh — Tue, 17 Jan 2017 12:50:37 GMT

It's nice to see you've solve your mipi-csi problem :smileyhappy:

Re: mipi csi2 can not receive data correctly! error on capturing video with ov9724 camera and imx6dl microcontroller.

navinars — Tue, 17 Jan 2017 13:12:41 GMT

Thanks for your help for solving the mipi-csi problem Wally :smileygrin:. Now I need to figure out how to make use of the frames I have saved.. :smileyhappy: