topic Re: imx6 support for 8-bit RAW camera in i.MX Processors

imx6 support for 8-bit RAW camera

ambiguity — Mon, 29 Apr 2013 21:44:30 GMT

Hi,

I am having trouble integrating our camera module & driver with the IPU. I have the majority of the driver complete (probe succeeds, i2c comm is good, I see dma debug (errors though)), however, I believe I am running into a pixel formatting / dma setup issue with the IPU driver. We expect (our User Layer app) to use V4L2_PIX_FMT_GREY, but this is "not supported". While investigating this issue, I continued to add cases to the mxc_v4l2_capture layer to accept V4L2_PIX_FMT_GREY, but the problems get deeper into the IPU layer.

Does imx6 support 8-bit RAW camera data? If so, what do I need to do?

Futher investigation into the IPU layer I discovered there is a "generic" format (or BAYER). I investigated a little... I believe a certain format is expected (header info maybe?) - but our CMOS sensor is outputting raw pixel data so I am not sure how I could use this format.

Here is a snippet of my debug:

In MVC:mxc_poll

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445611

case VIDIOC_DQBUF

In MVC:mxc_v4l_dqueue

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 40045613

case VIDIOC_STREAMOFF

In MVC:mxc_streamoff

_ipu_csi_wait4eof: channel = 335544276

_ipu_csi_wait4eof: IPU_IRQ_PRP_ENC_OUT_EOF

_ipu_csi_wait4eof: CSI stop timeout - 5 * 10ms

imx-ipuv3 imx-ipuv3.0: warning: disable ipu dma channel 20 during its busy state

Just before the ERROR the imager is capturing, but it just sits there an times out. I believe the camera/csi is setup correctly (vsync_pol, hsync_pol, etc.). I also see that the buffer(s) is (are) "queued".

Thanks!

Re: imx6 support for 8-bit RAW camera

rogerio_silva — Mon, 10 Jun 2013 17:49:53 GMT

Hi,

IPU can receive Grey image but it can't perform any kind of process like color space conversion, resize and etc because IPU will treat it as generic data, and not as a pixel.

You can receive this data (e.g. thorugh CSI) and handle it by softwares like gstreamer.

Rgds

Rogerio

Re: imx6 support for 8-bit RAW camera

ambiguity — Mon, 10 Jun 2013 18:20:06 GMT

Thank you Rogerio. I eventually figured it out. I added in all the support for IPU_PIX_FMT_GREY that pretty much followed IPU_PIX_FMT_GENERIC. The issue that caused all my problems was that by default, the Kernel takes on the path of CSI -> IC -> MEM (not CSI -> MEM). Once I figured that out... I was all set.

Re: imx6 support for 8-bit RAW camera

rogerio_silva — Mon, 10 Jun 2013 19:17:34 GMT

Very nice!

Thanks for the feedback.

Rogerio

Re: imx6 support for 8-bit RAW camera

richc128 — Thu, 13 Jun 2013 01:11:19 GMT

I'm running into the same problem as I want a pixel format of V4L2_PIX_FMT_GREY for my custom camera.

Can you tell me what files besides the camera driver you touched? I see a pattern in mxc_v4l2_capture.c where I could add a new GREY format, I see IPU_PIX_FMT_GREY in a lot of files...do I need to edit all of them?

Re: imx6 support for 8-bit RAW camera

ambiguity — Thu, 13 Jun 2013 13:33:31 GMT

Hi Richard, the files that I modified are listed below:

..\drivers\media\video\mxc\capture\ipu_csi_enc.c

..\drivers\media\video\mxc\capture\mxc_v4l2_capture.c

..\drivers\mxc\ipu3\ipu_capture.c

..\drivers\mxc\ipu3\ipu_common.c

..\drivers\mxc\ipu3\ipu_param_mem.h

Basically, where there is IPU_PIX_FMT_GENERIC, add another case for IPU_PIX_FMT_GREY. Also, in mxc_v4l2_capture.c:

-- You need to add support for V4L2_PIX_FMT_GREY if your camera driver is using/setting to that fmt. --

@@ -890,6 +995,10 @@ static int mxc_v4l2_s_fmt(cam_data *cam,

size = f->fmt.pix.width * f->fmt.pix.height * 3 / 2;

bytesperline = f->fmt.pix.width;

break;

+ case V4L2_PIX_FMT_GREY:

+ size = f->fmt.pix.width * f->fmt.pix.height;

+ bytesperline = f->fmt.pix.width;

+ break;

default:

break;

}

-- AND (per my previous post) -- :

Change mxc_capture_inputs to use "CSI MEM" depending on what input you are using. I #ifdef'd CONFIG_xxx it for now. If there is an easier way to do it... let me know. I just haven't had time to search where this is actually set.

Re: imx6 support for 8-bit RAW camera

DraganOstojic — Thu, 11 Jul 2013 19:43:30 GMT

I confirm that it works as you outlined. The only thing is that in my case I didn't have to change from CSI -> IC -> MEM to CSI -> MEM. In my case (imx6 sabresd imx6q board) kernel default path is always CSI -> MEM for both gray scale and color camera.

Thanks a lot for this explanation.

Re: imx6 support for 8-bit RAW camera

Selea — Tue, 23 Sep 2014 14:55:46 GMT

I doing the same thing and your answer hope we will solve it for mu board. Can you please clarify what you mean with

-- AND (per my previous post) -- :

thanks

Omar

Re: imx6 support for 8-bit RAW camera

ambiguity — Tue, 23 Sep 2014 15:07:51 GMT

Hi,

I believe I was referring to this portion of code in mxc_v4l2_capture.c:

static struct v4l2_input mxc_capture_inputs[MXC_V4L2_CAPTURE_NUM_INPUTS] = {

{

.index = 0,

#ifdef CONFIG_MXC_MYCAMERA

.name = "CSI MEM",

#else

.name = "CSI IC MEM",

#endif /*CONFIG_MXC_MYCAMERA*/

.type = V4L2_INPUT_TYPE_CAMERA,

.audioset = 0,

.tuner = 0,

.std = V4L2_STD_UNKNOWN,

.status = 0,

{

.index = 1,

.name = "CSI MEM",

.type = V4L2_INPUT_TYPE_CAMERA,

.audioset = 0,

.tuner = 0,

.std = V4L2_STD_UNKNOWN,

.status = V4L2_IN_ST_NO_POWER,

};

I believe it has to do with the .index value... I just rolled with the above change for my own testing purposes.

Re: imx6 support for 8-bit RAW camera

Selea — Tue, 23 Sep 2014 15:12:53 GMT

thanks,

I'm sure i will have more question, I'm trying to use a greyscale sensor on the two parallel port. But evrything was for color...

thanks to your thread i now have a way to follow....

Thanks a lot. I will disturb you more.

Omar

Re: imx6 support for 8-bit RAW camera

vinodmaverickr0 — Wed, 24 Sep 2014 07:29:58 GMT

Hi,

I'm running into the same problem as I want a pixel format of V4L2_PIX_FMT_SGBRG8 for my finger print sensor. I am using the imx6 sabresd board. I have connected the sensor to parallel port of IPU2(IPU1 w.r.t. to TRM).

I want to capture the image from sensor and send it to display.

Below is my implemetation in source code:

arch/arm/boot/dts/imx6q.dtsi

pinctrl_ipu2_2: ipu1grp-2 {

fsl,pins = <

MX6QDL_PAD_EIM_A18__IPU2_CSI1_DATA13 0x80000000

MX6QDL_PAD_EIM_A19__IPU2_CSI1_DATA14 0x80000000

MX6QDL_PAD_EIM_A20__IPU2_CSI1_DATA15 0x80000000

MX6QDL_PAD_EIM_A21__IPU2_CSI1_DATA16 0x80000000

MX6QDL_PAD_EIM_A22__IPU2_CSI1_DATA17 0x80000000

MX6QDL_PAD_EIM_A23__IPU2_CSI1_DATA18 0x80000000

MX6QDL_PAD_EIM_A24__IPU2_CSI1_DATA19 0x80000000

MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC 0x80000000

MX6QDL_PAD_EIM_A16__IPU2_CSI1_PIXCLK 0x80000000

MX6QDL_PAD_EIM_DA11__IPU2_CSI1_HSYNC 0x8000000

};

arch/arm/boot/dts/imx6qdl-sabresd.dtsi

regulators {

compatible = "simple-bus";

finger_3p3v: 3p3v { // custom supply for my sensor

compatible = "regulator-fixed";

regulator-name = "3P3V";

regulator-min-microvolt = <3300000>;

regulator-max-microvolt = <3300000>;

regulator-always-on;

}

v4l2_cap_2 { // To make the video node for the device which is connecte to ipu2 with CSI1

compatible = "fsl,imx6q-v4l2-capture";

ipu_id = <1>;

csi_id = <1>;

mclk_source = <0>;

status = "okay";

}

&i2c2{

clock-frequency = <100000>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_i2c3_2>;

status = "okay";

finger_print: finger_module@18 { // add the sensor entry having slave address 0x18

compatible = "fsl,finger-print";

reg = <0x18>;

VDDIO-supply = <&finger_3p3v>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_ipu2_2>;

clocks = <&clks 201>;

clock-names = "csi_mclk";

finger-gpio = <&gpio3 7 0>;

csi_id = <1>;

ipu_id = <1>;

mclk = <24000000>;

mclk_source = <0>;

}

arch/arm/mach-imx/mach-imx6q.c

if (of_machine_is_compatible("fsl,imx6q-sabresd") ||

of_machine_is_compatible("fsl,imx6q-sabreauto")){

regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 20, 1 << 20);

regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 17, 1 << 17);

regmap_update_bits(gpr, IOMUXC_GPR3, 2 << 6, 2 << 6);

regmap_update_bits(gpr, IOMUXC_GPR3, 1 << 10, 1 << 10);

}

When I probed the data line of sensor, It is sending data, as well as HSYNC and VSYNC.

I am facing the below errors:

imx6q-pinctrl 20e0000.iomuxc: pin MX6Q_PAD_CSI0_DAT12 already requested by 2-0018; cannot claim for 0-003c

root@imx6qsabresd:/bin# gst-launch mfw_v4lsrc device=/dev/video0 ! autovideosink

MAX resolution 1024x768

ERROR: v4l2 capture: slave not found!

MFW_GST_V4LSRC_PLUGIN 3.0.11 build on Sep 6 2014 12:31:36.

Setting pipeline to PAUSED ...

ERROR: v4l2 capture: slave not found!

MFW_GST_V4LSINK_PLUGIN 3.0

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

ERROR: from element /GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0: Internal data flow error.

Additional debug info:

/home/vinod/vvdn/VSTI_PoSX/yocto_src/build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.3:

streaming task paused, reason error (-5)

Execution ended after 10001604002 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

Setting pipeline to NULL ...

Total rendered:0

[--->FINALIZE v4l_sink

Freeing pipeline ...

root@imx6qsabresd:/bin#

Re: imx6 support for 8-bit RAW camera

Selea — Wed, 24 Sep 2014 07:47:11 GMT

the pin you are requsting is already used (requested ny another driver:

imx6q-pinctrl 20e0000.iomuxc: pin MX6Q_PAD_CSI0_DAT12 already requested by 2-0018; cannot claim for 0-003c

so maybe the driver is failing to load and so it is not creating all the stuff you need

ERROR: v4l2 capture: slave not found!

seem that is not creating the /dev/videoX device.

Re: imx6 support for 8-bit RAW camera

Selea — Wed, 24 Sep 2014 07:49:12 GMT

remember also the DATA_en pin, it should be pulled up (you have to define it as well in the pin list), otherwise the CSI doesn't capture.

Omar

Re: imx6 support for 8-bit RAW camera

Selea — Wed, 24 Sep 2014 08:24:55 GMT

now i understood what you mean on

static struct v4l2_input mxc_capture_inputs[MXC_V4L2_CAPTURE_NUM_INPUTS] = {

i'm working on your patch, so maybe if i can do it i will send you my idea so you can check. it

Re: imx6 support for 8-bit RAW camera

vinodmaverickr0 — Wed, 24 Sep 2014 11:19:42 GMT

Hi Omar,

Thanks for your reply.

In my DTSI file I have add:

v4l2_cap_2 { // To make the video node for the device which is connecte to ipu2 with CSI1

compatible = "fsl,imx6q-v4l2-capture";

ipu_id = <1>;

csi_id = <1>;

mclk_source = <0>;

status = "okay";

}

So it should create the node (/dev/video2) for IPU2 with CSI1. And my sensor device is attached on CSI1 of IPU2. So It should capture the data from video2 node and then send it to either display or memory.

As my sensor is continuously sending the data (which I have verified by probe the data line on oscilloscope). But I am not able to even capture the raw data, which is sending by the sensor.

I have not connected the data enable pin because there is no data enable pin in sensor side. So you are asking to enable it from the processor side. And if we will add it in the pin multiplexing. then how it will effect the working.

One more thing there are only 7 data lines, a HSYNC, VSYNC, and one pixel clock.

P.S. My sensor only support the Bayer RGB data format.

Re: imx6 support for 8-bit RAW camera

Selea — Thu, 25 Sep 2014 06:57:27 GMT

plase take a look at the files included here: (the dts and dtsi)

IPU2 parallel port on IMX6Q

this is my setup, is close to work... bu at least i see the dev/video3 (in my case) and i can acquire (forget the AR0134 driver... that is an old version with errors, i will upload the new one soon...)

Re: imx6 support for 8-bit RAW camera

vinodmaverickr0 — Thu, 25 Sep 2014 09:54:19 GMT

Hi,

I have also implemented the things in same way (which I have posted in earlier post).

But I have some doubts:

1) Is DATA_ENABLE pin must or not (because there is no data enable pin in my sensor). Right now data enable pin is low.

2) What should be the path for Bayer data (from CSI->IC->MEM or CSI->MEM).

Please upload the patch, when it will work.

Re: imx6 support for 8-bit RAW camera

Selea — Thu, 25 Sep 2014 11:55:05 GMT

the pin should be high otherwise tohave tochange it' spolarity in the IPU_IC_CONF

but this is not the reason why you do not see the device. ...

The strange is the ERROR "slave not found"

even if the data_en is wrong you should have read timeout error (the one I had) but the dev should be present...

even the path shoul not care.. but for the bayer the right one depends on what you want to do...

you can use CSI-> MEM so you will have the raw in memory, then maybe you have to do a color conversion...

but the first thing you have to fix is to have the V4L2 dev working and present... even if it will not be really acquiring.

Omar

Re: Re: imx6 support for 8-bit RAW camera

Selea — Thu, 25 Sep 2014 13:15:37 GMT

Hi,

i followed you patch.

but still I have a strange image captured.

this is the pipe i'm using:

gst-launch mfw_v4lsrc num-buffers=1 device=/dev/video3 ! filesink location=/mnt/nfs/home/sample.raw

the sensor is a 1280x960 greyscale so the capture should be 1228800 bytes.

the raw i have is 1800000 bytes it seems to be as a 4:2:0....

take a look at it, using gimp opening it as raw 1280x960.

Re: Re: imx6 support for 8-bit RAW camera

vinodmaverickr0 — Thu, 25 Sep 2014 14:21:56 GMT

Hi,

Can you please tell me how I can change the polarity.

I followed the below procedure, but I am not sure whether it is right or not.

drivers/mxc/ipu3/ipu_capture.c

data |= 0x82000300; // To change the polarity, In my case data enable pin is active high.

ipu_csi_write(ipu, csi, data, CSI_SENS_CONF);

I do not know, that why it is showing that slave is not found.

But in the imx6qd sabresd reference board, I face the same error, even when device is present and is working fine.So that is why I am ignoring this error.