imx6 support for 8-bit RAW camera

[ambiguity]

Hi,

I am having trouble integrating our camera module & driver with the IPU.  I have the majority of the driver complete (probe succeeds, i2c comm is good, I see dma debug (errors though)), however, I believe I am running into a pixel formatting / dma setup issue with the IPU driver.  We expect (our User Layer app) to use V4L2_PIX_FMT_GREY, but this is "not supported".  While investigating this issue, I continued to add cases to the mxc_v4l2_capture layer to accept V4L2_PIX_FMT_GREY, but the problems get deeper into the IPU layer.

Does imx6 support 8-bit RAW camera data?  If so, what do I need to do?

Futher investigation into the IPU layer I discovered there is a "generic" format (or BAYER).  I investigated a little... I believe a certain format is expected (header info maybe?) - but our CMOS sensor is outputting raw pixel data so I am not sure how I could use this format.

Here is a snippet of my debug:

In MVC:mxc_poll

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445611

   case VIDIOC_DQBUF

In MVC:mxc_v4l_dqueue

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 40045613

   case VIDIOC_STREAMOFF

In MVC:mxc_streamoff

_ipu_csi_wait4eof: channel = 335544276

_ipu_csi_wait4eof: IPU_IRQ_PRP_ENC_OUT_EOF

_ipu_csi_wait4eof: CSI stop timeout - 5 * 10ms

imx-ipuv3 imx-ipuv3.0: warning: disable ipu dma channel 20 during its busy state

Just before the ERROR the imager is capturing, but it just sits there an times out.  I believe the camera/csi is setup correctly (vsync_pol, hsync_pol, etc.).  I also see that the buffer(s) is (are) "queued".

Thanks!

[vinodmaverickr0]

Hi,

Can you please tell me how I can change the polarity.

I followed the below procedure, but I am not sure whether it is right or not.

drivers/mxc/ipu3/ipu_capture.c

data |= 0x82000300; // To change the polarity, In my case data enable pin is active high.

ipu_csi_write(ipu, csi, data, CSI_SENS_CONF);

I do not know, that why it is showing that slave is not found.

But in the imx6qd sabresd reference board, I face the same error, even when device is present and is working fine.So that is why I am ignoring this error.

[Selea]

it is the right place, if the DATA_en is pulled high you don't have to change anithing (bit=0 means active high), instead i your data_en is low you have to set the MSbit to 1.

data |= 0x80000000; /* i don't know why you set the other bits..)

in my code i did:

if(csi==1)

     cfg_param.data_en_pol=1;

but of course it is a ugly patch (i did the same more or less) it is because there is no a DATA_EN polarity  parameter in he DTS (such us for Hsync and Vsync and PLCK)...

so in the future we need to add is so it will be dynamic and not hard coded...

ARe you using the phyflex module?

Omar

[vinodmaverickr0]

Hi,

I have set the bit for below purpose:

1 << 31   // To invert the pin polarity, before it applied to internal circuitry. In my case DATA_EN pin is active high, so it means if I will set DATA_EN pin to 0, then after invert it will 1. So DATA_EN pin will be active at that time. After that only CSI circuit will capture the device data.

2<< 24  // So that data coming from CSI should first go to IC and then memory. I am not sure what value I have to define for it.

3<<8  // As my sensor module output format is BAYER, so I have set this bit to 3.

so (1<<31)|(2<<24)|(3<<8) = 0x82000300.

In my case it might be that DMA is not allocating to the IPU2. Because there is some thing like CSI_MEM0:CSIMEM3. For me it is looking like the memory for IPUs input ports.

It is very critical issue

So please suggest me, If you have nay idea.

Thanks.

[vinodmaverickr0]

Hi,

I'm running into the same problem as I want a pixel format of V4L2_PIX_FMT_SGBRG8 for my finger print sensor. I am using the imx6 sabresd board. I have connected the sensor to parallel port of IPU2(IPU1 w.r.t. to TRM).

I want to capture the image from sensor and send it to display.

Below is my implemetation in source code:

arch/arm/boot/dts/imx6q.dtsi

pinctrl_ipu2_2: ipu1grp-2 {

fsl,pins = <

MX6QDL_PAD_EIM_A18__IPU2_CSI1_DATA13  0x80000000

MX6QDL_PAD_EIM_A19__IPU2_CSI1_DATA14  0x80000000

MX6QDL_PAD_EIM_A20__IPU2_CSI1_DATA15  0x80000000

MX6QDL_PAD_EIM_A21__IPU2_CSI1_DATA16  0x80000000

MX6QDL_PAD_EIM_A22__IPU2_CSI1_DATA17  0x80000000

MX6QDL_PAD_EIM_A23__IPU2_CSI1_DATA18  0x80000000

MX6QDL_PAD_EIM_A24__IPU2_CSI1_DATA19  0x80000000

MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC 0x80000000

MX6QDL_PAD_EIM_A16__IPU2_CSI1_PIXCLK  0x80000000

MX6QDL_PAD_EIM_DA11__IPU2_CSI1_HSYNC 0x8000000

>;

};

arch/arm/boot/dts/imx6qdl-sabresd.dtsi

regulators {

          compatible = "simple-bus";

          finger_3p3v: 3p3v {  // custom supply for my sensor

               compatible = "regulator-fixed";

               regulator-name = "3P3V";

               regulator-min-microvolt = <3300000>;

               regulator-max-microvolt = <3300000>;

               regulator-always-on;

          }              

}

v4l2_cap_2 {  // To make the video node for the device which is connecte to ipu2 with CSI1

          compatible = "fsl,imx6q-v4l2-capture";

          ipu_id = <1>;

          csi_id = <1>;

          mclk_source = <0>;

          status = "okay";

}

&i2c2{

     clock-frequency = <100000>;

     pinctrl-names = "default";

     pinctrl-0 = <&pinctrl_i2c3_2>;

     status = "okay";

    finger_print: finger_module@18 {  // add the sensor entry having slave address 0x18

     compatible = "fsl,finger-print";

     reg = <0x18>;

     VDDIO-supply = <&finger_3p3v>;

     pinctrl-names = "default";

     pinctrl-0 = <&pinctrl_ipu2_2>;

     clocks = <&clks 201>;

     clock-names = "csi_mclk";

     finger-gpio = <&gpio3 7 0>;

     csi_id = <1>;

     ipu_id = <1>;

     mclk = <24000000>;

     mclk_source = <0>;

}

arch/arm/mach-imx/mach-imx6q.c

if (of_machine_is_compatible("fsl,imx6q-sabresd") ||

     of_machine_is_compatible("fsl,imx6q-sabreauto")){

          regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 20, 1 << 20);

          regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 17, 1 << 17);

          regmap_update_bits(gpr, IOMUXC_GPR3, 2 << 6, 2 << 6);

          regmap_update_bits(gpr, IOMUXC_GPR3, 1 << 10, 1 << 10);

}

When I probed the data line of sensor, It is sending data, as well as HSYNC and VSYNC.

I am facing the below errors:

imx6q-pinctrl 20e0000.iomuxc: pin MX6Q_PAD_CSI0_DAT12 already requested by 2-0018; cannot claim for 0-003c

root@imx6qsabresd:/bin# gst-launch mfw_v4lsrc device=/dev/video0 ! autovideosink

MAX resolution 1024x768

ERROR: v4l2 capture: slave not found!

MFW_GST_V4LSRC_PLUGIN 3.0.11 build on Sep  6 2014 12:31:36.

Setting pipeline to PAUSED ...

ERROR: v4l2 capture: slave not found!

MFW_GST_V4LSINK_PLUGIN 3.0

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

ERROR: from element /GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0: Internal data flow error.

Additional debug info:

/home/vinod/vvdn/VSTI_PoSX/yocto_src/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.3:

streaming task paused, reason error (-5)

Execution ended after 10001604002 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

Setting pipeline to NULL ...

Total rendered:0

[--->FINALIZE v4l_sink

Freeing pipeline ...

root@imx6qsabresd:/bin#

[mohamedsarjoond]

Hi vinod,

Out of your questions scope, just want to know the finger print sensor module which you used for your board. Can you please share the details ?

Thanks in advance.

Sarjoon.

[vinodmaverickr0]

Hi Mohamed,

Now It is working fine and I am able to get the finger print details like minutiae and ridge counting of the finger.

Thanks,

Vinod 

[Selea]

i think i found the problem: (referring to 3.10.17_1.0.1_ga)

v2f.fmt.pix.pixelformat is set to

cam->v2f.fmt.pix.pixelformat = V4L2_PIX_FMT_YUV420;

in the init_cam_struct function , and it is never updatewhit the value specified in the driver .

ar0134_data.pix.pixelformat = V4L2_PIX_FMT_GREY;

I added everywhere in the if or case my mew case (idenntical to V4L2_PIX_FORMA_GENERIC)

but it is not updated the cam->v2f.fmt.pix.pixelformat so it is alwasy V4L2_PIX_FMT_YUV420 and it does a big messy...

Omar

[vinodmaverickr0]

Hi,

It is true. But I have already change this in drivers/media/platform/mxc/capture/mxc_v4l2_capture.c

cam->v2f.fmt.pix.pixelformat = V4L2_PIX_FMT_SGBRG8

Apart from that I have also added the case of Bayer data format in some file

1) case V4L2_PIX_FMT_SGBRG8:

                        size = f->fmt.pix.width * f->fmt.pix.height;

                        bytesperline = f->fmt.pix.width;

2) if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_SGBRG8)

                pixel_fmt = IPU_PIX_FMT_GENERIC;

But still I am not able to see any captured data in the memory.

And now have no, Idea to resolve this issue

[Selea]

I think your error i somewhere before that step until you see the :

ERROR: v4l2 capture: slave not found!

the V4l2 device it is not accessed because not correctly linked , I think....

Omar

[vinodmaverickr0]

Hi,

Honestly I do not know why that error

ERROR: v4l2 capture: slave not found!

is not coming right now.

Either driver has find out the slave device or there is something wrong.

[vinodmaverickr0]

Hi,

I think clock for IPU2 is not initialized. So that is why it is not finding the device.

However I made some changes in u-boot level to initialize the clock for IPU2:

arch/arm/cpu/armv7/mx6/clock.c

void enable_ipu_clock(void){

reg |= (MXC_CCM_CCGR3_IPU2_IPU_MASK | MXC_CCM_CCGR3_IPU1_IPU_MASK);

}

board/freescale/mx6sabresd/mx6sabresd.c

static void setup_display(void){

reg |= (CHSCCDR_CLK_SEL_LDB_DI0

          << MXC_CCM_CHSCCDR_IPU2_DI0_CLK_SEL_OFFSET);

reg |= (CHSCCDR_CLK_SEL_LDB_DI0

          << MXC_CCM_CHSCCDR_IPU2_DI1_CLK_SEL_OFFSET);

}

drivers/video/ipu_common.c

static struct clk ipu_clk = {

.enable_shift = MXC_CCM_CCGR3_IPU2_IPU_DI0_OFFSET

}

But still I an not able to get any data in IPU2. I don't know should I have to also include the clock for IPU2 in kernel or not? 

[Selea]

remember also the DATA_en pin, it should be pulled up (you have to define it as well in the pin list), otherwise the CSI doesn't capture.

Omar

[Selea]

the pin you are requsting is already used (requested ny another driver:

imx6q-pinctrl 20e0000.iomuxc: pin MX6Q_PAD_CSI0_DAT12 already requested by 2-0018; cannot claim for 0-003c

so maybe the driver is failing to load and so it is not creating all the stuff you need

ERROR: v4l2 capture: slave not found!

seem that is not creating the /dev/videoX device.

[Selea]

thanks,

I'm sure i will have more question, I'm trying to use a greyscale sensor on the two parallel port. But evrything was for color...

thanks to your thread i now have a way to follow....

Thanks a lot. I will disturb you more.

Omar

[DraganOstojic]

I confirm that it works as you outlined. The only thing is that in my case I didn't have to change from CSI -> IC -> MEM to CSI -> MEM. In my case (imx6 sabresd imx6q board) kernel default path is always CSI -> MEM for both gray scale and color camera.

Thanks a lot for this explanation.

[rogerio_silva · Accepted solution]

Hi,

IPU can receive Grey image but it can't perform any kind of process like color space conversion, resize and etc because IPU will treat it as generic data, and not as a pixel.

You can receive this data (e.g. thorugh CSI) and handle it by softwares like gstreamer.

Rgds

Rogerio

[ambiguity]

Thank you Rogerio.  I eventually figured it out.  I added in all the support for IPU_PIX_FMT_GREY that pretty much followed IPU_PIX_FMT_GENERIC.  The issue that caused all my problems was that by default, the Kernel takes on the path of CSI -> IC -> MEM (not CSI -> MEM).  Once I figured that out... I was all set.

[rogerio_silva]

Very nice!

Thanks for the feedback.

Rogerio