Linux Drivers for Aptina AR0331(parallel interface) on iMX6D.

[zaheerm]

Would like to know if any one has attempted to develop drivers for aptina AR0331 image sensor for iMX6D. I am using linux-3.10.9 kernel.

Which sensor can be used as a reference to begin with? Any pointers are welcome.

[jimmychan]

the Linux BSP documentation may useful for you. You can download it from here:

< http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=IMX6_SW >

Under Documentation -- >

• Linux Bundle for i.MX 6Quad/Dual

For reference, please read the i.MX_6Dual6Quad_BSP_Porting_Guide.pdf and i.MX_6Dual6Quad_Linux_Reference_Manual.pdf .

[zaheerm]

Hi,

I am facing some problem while receiving RGB Bayer video data from 12-bit (parallel) aptina sensor.

In summary, the application times-out while DQ operation:

In MVC:mxc_v4l_dqueue

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

When I probe the lines from the sensor, I am able to see activity on the HSYNC, VSYNC, PIXCLK and DATA.

Setup details:

Out of 12 data lines, 2 lines are not connected, effectively making it a 10-bit interface.

The LOG while running the 'yavta' application is as below: ./yavta -i 1  -f SBGGR10 -c=10 -s 1280x720 -n 2 /dev/video0

********************************************************************

In MVC: mxc_v4l_open

   device name is Mxc Camera

End of mxc_v4l_open: v2f pix widthxheight 1280 x 720

End of mxc_v4l_open: crop_bounds widthxheight 1280 x 720

End of mxc_v4l_open: crop_defrect widthxheight 1280 x 720

End of mxc_v4l_open: crop_current widthxheight 1280 x 720

On Open: Input to ipu size is 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 80685600

   case VIDIOC_QUERYCAP

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0045627

   case VIDIOC_S_INPUT

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 80045626

   case VIDIOC_G_INPUT

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0cc5605

   case VIDIOC_S_FMT

- In MVC: mxc_v4l2_s_fmt

   type=V4L2_BUF_TYPE_VIDEO_CAPTURE

End of mxc_v4l2_s_fmt: v2f pix widthxheight 1280 x 720

End of mxc_v4l2_s_fmt: crop_bounds widthxheight 1280 x 720

End of mxc_v4l2_s_fmt: crop_defrect widthxheight 1280 x 720

End of mxc_v4l2_s_fmt: crop_current widthxheight 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0cc5604

   case VIDIOC_G_FMT

In MVC: mxc_v4l2_g_fmt type=1

   type is V4L2_BUF_TYPE_VIDEO_CAPTURE

End of mxc_v4l2_g_fmt: v2f pix widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_bounds widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_current widthxheight 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0145608

   case VIDIOC_REQBUFS

In MVC:mxc_streamoff

MVC: In mxc_free_frame_buf

In MVC:mxc_allocate_frame_buf - size=1843200

cma: dma_alloc_from_contiguous(cma dc0e4ec0, count 450, align 8)

cma: dma_alloc_from_contiguous(): returned 81cd2000

cma: dma_alloc_from_contiguous(cma dc0e4ec0, count 450, align 8)

cma: dma_alloc_from_contiguous(): returned 81cd6000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

   case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

   pgoff=0x76400, start=0x76cd2000, end=0x76e94000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

   case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

   pgoff=0x76600, start=0x76b10000, end=0x76cd2000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

   case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

   case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 40045612

   case VIDIOC_STREAMON

In MVC:mxc_streamon

IPU:In csi_enc_enabling_tasks

cma: dma_alloc_from_contiguous(cma dc0e4ec0, count 450, align 8)

cma: dma_alloc_from_contiguous(): returned 81cda000

In csi_enc_setup

imx-ipuv3 2800000.ipu: init channel = 15

imx-ipuv3 2800000.ipu: ipu busfreq high requst.

imx-ipuv3 2800000.ipu: IPU_CONF =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_CONF =     0x0000002F

imx-ipuv3 2800000.ipu: IDMAC_CHA_EN1 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_CHA_EN2 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_CHA_PRI1 =     0x18800001

imx-ipuv3 2800000.ipu: IDMAC_CHA_PRI2 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_BAND_EN1 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_BAND_EN2 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_DB_MODE_SEL0 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_DB_MODE_SEL1 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_TRB_MODE_SEL0 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_TRB_MODE_SEL1 =     0x00000000

imx-ipuv3 2800000.ipu: DMFC_WR_CHAN =     0x00000090

imx-ipuv3 2800000.ipu: DMFC_WR_CHAN_DEF =     0x202020F6

imx-ipuv3 2800000.ipu: DMFC_DP_CHAN =     0x00009694

imx-ipuv3 2800000.ipu: DMFC_DP_CHAN_DEF =     0x2020F6F6

imx-ipuv3 2800000.ipu: DMFC_IC_CTRL =     0x00000002

imx-ipuv3 2800000.ipu: IPU_FS_PROC_FLOW1 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_FS_PROC_FLOW2 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_FS_PROC_FLOW3 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_FS_DISP_FLOW1 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_VDIC_VDI_FSIZE =     0x00000000

imx-ipuv3 2800000.ipu: IPU_VDIC_VDI_C =     0x00000000

imx-ipuv3 2800000.ipu: IPU_IC_CONF =     0x40000000

setting IDMAC channel parameters for GENERIC-16 mode

imx-ipuv3 2800000.ipu: initializing idma ch 0 @ f0180000

!!! : setting RWS enable for generic sensor

imx-ipuv3 2800000.ipu: IPU_CONF =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_CONF =     0x0000002F

imx-ipuv3 2800000.ipu: IDMAC_CHA_EN1 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_CHA_EN2 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_CHA_PRI1 =     0x18800001

imx-ipuv3 2800000.ipu: IDMAC_CHA_PRI2 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_BAND_EN1 =     0x00000000

imx-ipuv3 2800000.ipu: IDMAC_BAND_EN2 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_DB_MODE_SEL0 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_DB_MODE_SEL1 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_TRB_MODE_SEL0 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_CHA_TRB_MODE_SEL1 =     0x00000000

imx-ipuv3 2800000.ipu: DMFC_WR_CHAN =     0x00000090

imx-ipuv3 2800000.ipu: DMFC_WR_CHAN_DEF =     0x202020F6

imx-ipuv3 2800000.ipu: DMFC_DP_CHAN =     0x00009694

imx-ipuv3 2800000.ipu: DMFC_DP_CHAN_DEF =     0x2020F6F6

imx-ipuv3 2800000.ipu: DMFC_IC_CTRL =     0x00000002

imx-ipuv3 2800000.ipu: IPU_FS_PROC_FLOW1 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_FS_PROC_FLOW2 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_FS_PROC_FLOW3 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_FS_DISP_FLOW1 =     0x00000000

imx-ipuv3 2800000.ipu: IPU_VDIC_VDI_FSIZE =     0x00000000

imx-ipuv3 2800000.ipu: IPU_VDIC_VDI_C =     0x00000000

imx-ipuv3 2800000.ipu: IPU_IC_CONF =     0x40000000

imx-ipuv3 2800000.ipu: ch 0 word 0 - 00000000 00000000 00000000 E0001800 000B3C9F

imx-ipuv3 2800000.ipu: ch 0 word 1 - 0ED00000 01DA0000 00C7C000 00027FC0 00000000

imx-ipuv3 2800000.ipu: PFS 0x6,

imx-ipuv3 2800000.ipu: BPP 0x3,

imx-ipuv3 2800000.ipu: NPB 0x1f

imx-ipuv3 2800000.ipu: FW 1279,

imx-ipuv3 2800000.ipu: FH 719,

imx-ipuv3 2800000.ipu: EOL1 0,

imx-ipuv3 2800000.ipu: EBA0 0x76800000

imx-ipuv3 2800000.ipu: EBA1 0x76800000

imx-ipuv3 2800000.ipu: Stride 2559

imx-ipuv3 2800000.ipu: scan_order 0

imx-ipuv3 2800000.ipu: uv_stride 0

imx-ipuv3 2800000.ipu: u_offset 0x0

imx-ipuv3 2800000.ipu: v_offset 0x0

imx-ipuv3 2800000.ipu: Width0 0+1,

imx-ipuv3 2800000.ipu: Width1 0+1,

imx-ipuv3 2800000.ipu: Width2 0+1,

imx-ipuv3 2800000.ipu: Width3 0+1,

imx-ipuv3 2800000.ipu: Offset0 0,

imx-ipuv3 2800000.ipu: Offset1 0,

imx-ipuv3 2800000.ipu: Offset2 0,

imx-ipuv3 2800000.ipu: Offset3 0

eba 76400000

eba 76600000

IPU_CONF     0x00000102

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445611

   case VIDIOC_DQBUF

In MVC:mxc_v4l_dqueue

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0145608

   case VIDIOC_REQBUFS

In MVC:mxc_streamoff

imx-ipuv3 2800000.ipu: CSI stop timeout - 5 * 10ms

imx-ipuv3 2800000.ipu: ipu busfreq high release.

cma: dma_release_from_contiguous(page 81cda000)

In MVC:mxc_free_frames

MVC: In mxc_free_frame_buf

cma: dma_release_from_contiguous(page 81cd2000)

cma: dma_release_from_contiguous(page 81cd6000)

In MVC:mxc_allocate_frame_buf - size=1843200

In MVC:mxc_v4l_close

In MVC:mxc_streamoff

mxc_v4l_close: release resource

MVC: In mxc_free_frame_buf

In MVC:mxc_free_frames

power_down_callback

********************************************************************

I have added the required support for bayer in the sources as per the discussion:

https://community.freescale.com/message/353698?et=watches.email.thread#353698

The sensor is connected to CSI1-IPU2.

My DTS has the following configuration:

ipu2: ipu@02800000 {

    compatible = "fsl,imx6q-ipu";

        reg = <0x02800000 0x400000>;

        interrupts = <0 8 0x4 0 7 0x4>;

        clocks = <&clks 133>, <&clks 134>, <&clks 137>,

                 <&clks 41>, <&clks 42>,

                 <&clks 135>, <&clks 136>;

        clock-names = "bus", "di0", "di1",

                 "di0_sel", "di1_sel",

                 "ldb_di0", "ldb_di1";

                 resets = <&src 4>;

                 bypass_reset = <0>;

};

&iomuxc {

        ipu2 {

                pinctrl_ipu2_1: ipu2grp-1 { (Parallel) Image Sensor */

                        fsl,pins = <

                                MX6QDL_PAD_EIM_DA11__IPU2_CSI1_HSYNC    0x80000000

                                MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC    0x80000000

                                MX6QDL_PAD_EIM_A16__IPU2_CSI1_PIXCLK    0x80000000

                                MX6QDL_PAD_EIM_DA9__IPU2_CSI1_DATA00    0x80000000

                                MX6QDL_PAD_EIM_DA8__IPU2_CSI1_DATA01    0x80000000

                                MX6QDL_PAD_EIM_DA7__IPU2_CSI1_DATA02    0x80000000

                                MX6QDL_PAD_EIM_DA6__IPU2_CSI1_DATA03    0x80000000

                                MX6QDL_PAD_EIM_DA5__IPU2_CSI1_DATA04    0x80000000

                                MX6QDL_PAD_EIM_DA4__IPU2_CSI1_DATA05    0x80000000

                                MX6QDL_PAD_EIM_DA3__IPU2_CSI1_DATA06    0x80000000

                                MX6QDL_PAD_EIM_DA2__IPU2_CSI1_DATA07    0x80000000

                                MX6QDL_PAD_EIM_DA1__IPU2_CSI1_DATA08    0x80000000

                                MX6QDL_PAD_EIM_DA0__IPU2_CSI1_DATA09    0x80000000

                                MX6QDL_PAD_CSI0_DAT16__GPIO6_IO02       0x80000000   

                                MX6QDL_PAD_CSI0_VSYNC__GPIO5_IO21       0x80000000  

                                MX6QDL_PAD_CSI0_DATA_EN__GPIO5_IO20     0x80000000 

                        >;

                };

        };

v4l2_cap_0 {

      compatible = "fsl,imx6q-v4l2-capture";

      ipu_id = <1>;

      csi_id = <1>;

      mclk_source = <0>;

       status = "okay";

};

&i2c1 {

        status = "okay";

        clock-frequency = <100000>;

        pinctrl-names = "default";

        pinctrl-0 = <&pinctrl_i2c1_2>;

        ar0331: ar0331@10 {

                compatible = "ar0331";

                reg = <0x10>;

                pinctrl-names = "default";

                pinctrl-0 = <&pinctrl_ipu2_1>;

                clock-names = "csi_mclk";

                ipu_id = <1>;

                csi_id = <1>;

                mclk = <24000000>;

                mclk_source = <0>;

                rst-gpios = <&gpio6 2 1>;           

        };

};

[qiang_li-mpu_se]

Hi Zaheer M, what's your sensor's output format and how about the frame length? Please also dump the IPU2_CSI1_SENS_CONF, IPU2_CSI1_SENS_FRM_SIZE, IPU2_CSI1_ACT_FRM_SIZE and IPU2_SMFC_BS registers.

From your IDMAC setting, for each frame, the CSI needs receive 1280*720*2 bytes.

By the way, your data line connection IPU2_CSI1_DATA09 ~ IPU2_CSI1_DATA00 for 10 bits CSI interface is also wrong, you shoudl use IPU2_CSI1_DATA19 ~ IPU2_CSI1_DATA10.

[zaheerm]

For the data-line connections, the reference is in this section i.e. 37.4.3.2.1 Parallel Interface of the RM.

[zaheerm]

Hi Qiang,

Thanks for responding to the query.

You had mentioned the calculation of 'Frame size' in an earlier post, can you pl. tell me what values would be used for receiving 1280x720 bayer frames.

You have mentioned that we need to use Data-19  to Data-10 for receiving 10-bit data, Can you also mention how to arrive at using which data lines for receiving 10-bit data from the sensor. Is there a mention of it in the data-sheet or the reference manual?

Thanks,

Zaheer.

[qiang_li-mpu_se]

Yes, the CSI data line MSB information comes from reference manual 37.4.3.2.1 Parallel Interface.

For your setting, the frame size is 1280*720*2, I don't think it is enough for a 10 bits bayer input. If the bayer camera sensor will send 10 bits R, G, B for a pixel, then from CSI side of view, for each pixel, it needs 6 bytes, so the the frame size should be 1280*720*6; and if you use 16 bits generic mode to capture the data, in IDMAC and CSI registers, you can set bpp=16, width = (1280*3) and height = 720; then the CSI can receice a correct frame in (1280*3)*720*2 bytes.

[zaheerm]

How do i configure the register "IOMUXC_IPU2_SENS1_VSYNC_SELECT_INPUT" in linux-3.10.9 kernel to select the Daisy Field?

[qiang_li-mpu_se]

You can seach "MX6QDL_PAD_EIM_D29__IPU2_CSI1_VSYNC" or "MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC" in in file "/arch/arm/boot/dts/imx6q-pinfunc.h"., the select input value had already been defined there.

You can add the IOMUX define into "imx6q.dtsi".

[zaheerm]

My "imx6q.dtsi" file already has the PAD_PIN configuration, as below. I don't have any sample code for configuration of the input register i.e. "IOMUXC_IPU2_SENS1_VSYNC_SELECT_INPUT"

&iomuxc {

        ipu2 {

                pinctrl_ipu2_1: ipu2grp-1 { (Parallel) Image Sensor */

                        fsl,pins = <

                                MX6QDL_PAD_EIM_DA11__IPU2_CSI1_HSYNC    0x80000000

                                MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC    0x80000000

                                MX6QDL_PAD_EIM_A16__IPU2_CSI1_PIXCLK    0x80000000

                                MX6QDL_PAD_EIM_DA9__IPU2_CSI1_DATA00    0x80000000

                                MX6QDL_PAD_EIM_DA8__IPU2_CSI1_DATA01    0x80000000

                                MX6QDL_PAD_EIM_DA7__IPU2_CSI1_DATA02    0x80000000

                                MX6QDL_PAD_EIM_DA6__IPU2_CSI1_DATA03    0x80000000

                                MX6QDL_PAD_EIM_DA5__IPU2_CSI1_DATA04    0x80000000

                                MX6QDL_PAD_EIM_DA4__IPU2_CSI1_DATA05    0x80000000

                                MX6QDL_PAD_EIM_DA3__IPU2_CSI1_DATA06    0x80000000

                                MX6QDL_PAD_EIM_DA2__IPU2_CSI1_DATA07    0x80000000

                                MX6QDL_PAD_EIM_DA1__IPU2_CSI1_DATA08    0x80000000

                                MX6QDL_PAD_EIM_DA0__IPU2_CSI1_DATA09    0x80000000

                                MX6QDL_PAD_CSI0_DAT16__GPIO6_IO02       0x80000000   

                                MX6QDL_PAD_CSI0_VSYNC__GPIO5_IO21       0x80000000  

                                MX6QDL_PAD_CSI0_DATA_EN__GPIO5_IO20     0x80000000 

                        >;

                };

        };

[qiang_li-mpu_se · Accepted solution]

You don't need set IOMUXC_IPU2_SENS1_VSYNC_SELECT_INPUT alone, it was covered in MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC, when the BSP initializing the PIN EIM_DA12 as CSI1_VSYNC, the IOMUXC_IPU2_SENS1_VSYNC_SELECT_INPUT will be set too.

From the define, we can know the IOMUXC_IPU2_SENS1_VSYNC_SELECT_INPUT register will be set to 0x1.

/*

* The pin function ID is a tuple of

* <mux_reg conf_reg input_reg mux_mode input_val>

*/

#define MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC        0x144 0x458 0x8e4 0x2 0x1

[zaheerm]

Thanks for the info.

I still don't have the EOF being triggered while capturing the data. Can you guide me which registers i.e. IPU/CSI/IDMAC need to be check to identify the cause of missing EOF.

[Selea]

similar problem. Did you solved?

IPU2 parallel port on IMX6Q

please share if you got the solution.

Thanks

Omar

[ko-hey]

Dear Zaheer M

Hi

Have you solved this issue?

My customer want to use same sensor.

So if you solve this issue please share me.

Best Regards,

Ko-hey

[qiang_li-mpu_se]

If the sensor's real frame data is less than the frame size setting in CSI register and IDMAC, there will be no EOF, because CSI can't capture enough data for a frame.