IPU2 parallel port on IMX6Q

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

IPU2 parallel port on IMX6Q

9,088 Views
Selea
Senior Contributor I

Dear All,

 

Now we are working to our custom board. The hardware is close to nitrogen6x board from boundary, but with the phytec flexboard as module on it.

We would like to use the second parallel sensor board because we already have another sensor (well a video converter) on the first parallel (IPU1/CSI0)

so we need another parallel sensor interface.

we are using as a base the 3.10.17-1.0.1-GA

 

I wrote my own driver for the sensor we planned to use , but basically it is the same of ov5640.c (parallel version)

i modify the DTS files adding our sensor and using the nitrogen6x as reference with minor change on pin mux.

from DTS:

pin muxing:

 

ar0134  {
     pinctrl_ar0134: pinctrl_ar0134 {
   fsl,pins = <
   MX6QDL_PAD_EIM_A17__IPU2_CSI1_DATA12    0xb0b1
   MX6QDL_PAD_EIM_A18__IPU2_CSI1_DATA13    0xb0b1
   MX6QDL_PAD_EIM_A19__IPU2_CSI1_DATA14    0xb0b1
   MX6QDL_PAD_EIM_A20__IPU2_CSI1_DATA15    0xb0b1
   MX6QDL_PAD_EIM_A21__IPU2_CSI1_DATA16    0xb0b1
   MX6QDL_PAD_EIM_A22__IPU2_CSI1_DATA17    0xb0b1
   MX6QDL_PAD_EIM_A23__IPU2_CSI1_DATA18    0xb0b1
   MX6QDL_PAD_EIM_A24__IPU2_CSI1_DATA19    0xb0b1
   MX6QDL_PAD_EIM_DA10__IPU2_CSI1_DATA_EN    0xb0b1
   MX6QDL_PAD_EIM_A16__IPU2_CSI1_PIXCLK    0xb0b1
   MX6QDL_PAD_EIM_DA11__IPU2_CSI1_HSYNC    0xb0b1
   MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC    0xb0b1
   >;
   };

    };

 

from DTS add the sensor on I2c:

 

&i2c3 {

    pinctrl-names = "default";

    pinctrl-0 = <&pinctrl_i2c3>;

    clock-frequency = <100000>;

    status = "okay";

 

    ar0134: ar0134@10 {

        compatible = "apti,ar0134";

        reg = <0x10>;

        pinctrl-names = "default";

        pinctrl-0 = <&pinctrl_ar0134>;

        //pinctrl-0 = <&pinctrl_ipu1_2>;

        clocks = <&clk27m 0>;

        clock-names = "csi_mclk";

        //DOVDD-supply = <&vgen4_reg>;

        //AVDD-supply = <&vgen3_reg>; 

        //DVDD-supply = <&vgen2_reg>; 

        //pwn-gpios = <&gpio1 16 1>;  

        //rst-gpios = <&gpio1 17 0>;  

        ipu_id = <1>;

        csi_id = <1>;

        mclk = <27000000>;

        mclk_source = <0>;

    };

};

 

 

the driver is loaded correctly, it see the sensor and the v4l2 create the /dev/video3 , when i try to acquire something from that device (with gstreamer or just cat it).

I always have back "ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0"

 

attached here you can see the DTS files I'm using and also the driver.

it seems the something is still not configured proprely on the second CSI parallel port so it dosn't acquire.

 

the signal from the sensor are correct (Vsync,Hsync,pixclk) and also the data_en seems to be ok, i tried also to negate it in the SENS_CONF register of the IPU_CSI

I would like to know which are the changes needed to be made in order to get IPU2/CSI1 working as the device is being correctly loaded, however it cannot be feed by any frame.

 

Thanks to all

Omar

Original Attachment has been moved to: ar0134.c.zip

Original Attachment has been moved to: Kconfig.zip

Original Attachment has been moved to: Makefile.zip

Original Attachment has been moved to: imx6q-phytec-pbab01.dts.zip

Original Attachment has been moved to: imx6qdl-phytec-pfla02.dtsi.zip

Labels (4)
21 Replies

2,536 Views
Selea
Senior Contributor I

just adding some log that shows the error i have back:

clock_curr=mclk=27000000

ipu_csi_window_size_crop: Error left=0 top=59455247

ipu_csi_window_size_crop: dopo Error left=280 top=1e0

power_up_camera: ipu1/csi1

ioctl_init

Setting mclk to 27 MHz

AR0134_init  

clock_curr=mclk=27000000

ipu_csi_window_size_crop: Error left=0 top=59455247

ipu_csi_window_size_crop: dopo Error left=280 top=1e0

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

power_down_callback: ipu1/csi1

i will enable more pr_debug to have a more verbose log.

0 Kudos

2,536 Views
Selea
Senior Contributor I

Something more about my test:

- Ican say DATA_EN is used also in parallel port so is missing a parameters to configure data_en polarity from the sensor driver such as vert and horiz sync :

p->u.bt656.nobt_vs_inv = 1;

p->u.bt656.nobt_hs_inv = 1;

so maybe we need to add a

p->u.bt656.nobt_dataen_inv = 1;

now i need to understand how to configure the sensor port on how to capture a grey scale sensor :YYYYYYYYYYYYYYYYYY

V4L2_PIX_FMT_GREY is not handled by ipu_capture.c

Now i don't see the timeout error anymore, even if the image captured is not correct yet... but the sensor port seems to acquire something

Thanks

Omar

0 Kudos

2,536 Views
Selea
Senior Contributor I

so I added the support for IPU_PIX_FMT_GREY (in the generic)

ipu_csi_init_interface(struct ipu_soc *ipu, uint16_t width, uint16_t height,

    uint32_t pixel_fmt, ipu_csi_signal_cfg_t cfg_param)

{

    uint32_t data = 0;

    uint32_t csi = cfg_param.csi;

   

    /* Set SENS_DATA_FORMAT bits (8, 9 and 10)

       RGB or YUV444 is 0 which is current value in data so not set

       explicitly

       This is also the default value if attempts are made to set it to

       something invalid. */

    switch (pixel_fmt) {

    case IPU_PIX_FMT_YUYV:

        cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_YUV422_YUYV;

        break;

    case IPU_PIX_FMT_UYVY:

        cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_YUV422_UYVY;

        break;

    case IPU_PIX_FMT_RGB24:

    case IPU_PIX_FMT_BGR24:

        cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_RGB_YUV444;

        break;

    case IPU_PIX_FMT_GENERIC:

    case IPU_PIX_FMT_GENERIC_16:

    case IPU_PIX_FMT_GREY:     

        cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_BAYER;

        break;

    case IPU_PIX_FMT_RGB565:

        cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_RGB565;

        break;

    case IPU_PIX_FMT_RGB555:

        cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_RGB555;

        break;

    default:

        dev_dbg(ipu->dev, "%s:pixel_fmt=%x\n", __func__, pixel_fmt);

        return -EINVAL;

    }

/* to patch my personal HW */

if(csi==1)

     cfg_param.data_en_pol=1;

but we nned to add a new field. to support the dinamic configuration of data_en polarity from DTS

0 Kudos

2,536 Views
Selea
Senior Contributor I

this seems to be the solution for the raw acquisition:

imx6 support for 8-bit RAW camera

0 Kudos

2,536 Views
Selea
Senior Contributor I

the solution is still not working....

still I have a strange image captured.

this is the pipe i'm using:

gst-launch mfw_v4lsrc num-buffers=1 device=/dev/video3 ! filesink location=/mnt/nfs/home/sample.raw

the sensor is a 1280x960 greyscale so the capture should be 1228800 bytes.

the raw i have is 1800000 bytes it seems to be as a 4:2:0....

take a look at it, using gimp opening it as raw 1280x960.

0 Kudos

2,536 Views
Selea
Senior Contributor I

i think i found the problem: (referring to 3.10.17_1.0.1_ga)

v2f.fmt.pix.pixelformat is set to

cam->v2f.fmt.pix.pixelformat = V4L2_PIX_FMT_YUV420;

in the init_cam_struct function , and it is never updatewhit the value specified in the driver .

ar0134_data.pix.pixelformat = V4L2_PIX_FMT_GREY;

I added everywhere in the "if" or "case" my mew case (idenntical to V4L2_PIX_FMT_GENERIC)

but it is not updated anywhere the cam->v2f.fmt.pix.pixelformat so it is alwasy V4L2_PIX_FMT_YUV420 and it does a big messy...

Omar

0 Kudos

2,536 Views
Selea
Senior Contributor I

I'm really diggin iside CSI-IPU- chain.

I can say that there are some missing control and "case"... Mainly it works if the sensor come in as YUV422 and the output from the ipu is YUV420....

any other case (Bayer ...  grey scale... )  are not considered so they are not working proprely.

So i think the CSI ---> IPU ---> chain enned a "serious" patch to include the missing cases.... I think the most common cases could be sensor with Bayer output and greyscale output.

Omar

0 Kudos

2,536 Views
Selea
Senior Contributor I

Important FIX:

in file ipu_still.c

static int prp_still_start(void *private)

{

    cam_data *cam = (cam_data *) private;

    u32 pixel_fmt;

    int err;

    ipu_channel_params_t params;

    pr_err("%s:ipu_channel_request still %d %d \n", __func__,cam->ipu_id,cam->csi);   

   

    if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_YUV420)

        pixel_fmt = IPU_PIX_FMT_YUV420P;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_NV12)

        pixel_fmt = IPU_PIX_FMT_NV12;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_YUV422P)

        pixel_fmt = IPU_PIX_FMT_YUV422P;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_UYVY)

        pixel_fmt = IPU_PIX_FMT_UYVY;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_YUYV)

        pixel_fmt = IPU_PIX_FMT_YUYV;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_BGR24)

        pixel_fmt = IPU_PIX_FMT_BGR24;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_RGB24)

        pixel_fmt = IPU_PIX_FMT_RGB24;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_RGB565)

        pixel_fmt = IPU_PIX_FMT_RGB565;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_BGR32)

        pixel_fmt = IPU_PIX_FMT_BGR32;

    else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_RGB32)

        pixel_fmt = IPU_PIX_FMT_RGB32;

    else {

        printk(KERN_ERR "format not supported\n");

        return -EINVAL;

    }

    memset(&params, 0, sizeof(params));

    params.csi_mem.csi = cam->csi;  // OMAR : otherwise the CSI number is missing from here and forever

   

    err = ipu_channel_request(cam->ipu, CSI_MEM, &params, &cam->ipu_chan);

    if (err) {

        pr_err("%s:ipu_channel_request %d\n", __func__, err);

        return err;

    }

    err = ipu_init_channel_buffer(cam->ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

                      pixel_fmt, cam->v2f.fmt.pix.width,

                      cam->v2f.fmt.pix.height,

                      cam->v2f.fmt.pix.width, IPU_ROTATE_NONE,

                      cam->still_buf[0], cam->still_buf[1], 0,

                      0, 0);

    if (err != 0)

        return err;

#ifdef CONFIG_MXC_IPU_V1

    ipu_clear_irq(IPU_IRQ_SENSOR_OUT_EOF);

    err = ipu_request_irq(IPU_IRQ_SENSOR_OUT_EOF, prp_still_callback,

                  0, "Mxc Camera", cam);

    if (err != 0) {

        printk(KERN_ERR "Error registering irq.\n");

        return err;

    }

    callback_flag = 0;

    callback_eof_flag = 0;

    ipu_clear_irq(IPU_IRQ_SENSOR_EOF);

    err = ipu_request_irq(IPU_IRQ_SENSOR_EOF, prp_csi_eof_callback,

                  0, "Mxc Camera", cam);

    if (err != 0) {

        printk(KERN_ERR "Error IPU_IRQ_SENSOR_EOF\n");

        return err;

    }

#else

    callback_eof_flag = 0;

    buffer_num = 0;

    ipu_clear_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF);

    err = ipu_request_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF,

                  prp_still_callback,

                  0, "Mxc Camera", cam);

    if (err != 0) {

        printk(KERN_ERR "Error registering irq.\n");

        return err;

    }

    ipu_select_buffer(cam->ipu, CSI_MEM, IPU_OUTPUT_BUFFER, 0);

    ipu_enable_channel(cam->ipu, CSI_MEM);

    cam_ipu_enable_csi(cam);

#endif

    return err;

}

0 Kudos

2,533 Views
bobosv
Contributor II

Any updates?

I'm currently viewing similar troubles trying to enable IPU2 CSI1 interface

/B

0 Kudos

2,532 Views
bobosv
Contributor II

Found it, The IOMUX_GPR1 register bit 19-20 needed to be set to select Paralell CSI interface!!

/B

2,536 Views
Selea
Senior Contributor I

It is important for me to understand the configuration of the polarity of sync signal.It is not clear at all in the reference manual.

in the SENS_CONF register description they say just if you write a "1" in that bit the signal is inverted before pass it to the internal circuitry...

Ok but means active high or active low?

I'm trying with all possible settings, but could be better to know it.

my signal are:

Vsync active high (mens during high period of Vsync i have valid lines)

Hsync active high(during high period of Hsync I have valid data)

Pclk active high (data valid on rising edge)

Data_en is used in parallel CSI?or not?

Thanks

Omar

0 Kudos

2,536 Views
Selea
Senior Contributor I

an update:

the sensor AR0134 is a BW sensor so the output is YYYYYYY and not YUYV. SO in my driver there wea an error:

//ar0134_data.pix.pixelformat = V4L2_PIX_FMT_YUYV;

    ar0134_data.pix.pixelformat = V4L2_PIX_FMT_GREY;

but still i have the same error

just to ASK... the DATA_EN is used or not in parallel mode in the CSI (i read different ideas on different threads)? On my hardware is pulled low,

i tried to set the bit 31 in the SENS_CONF register but nothing change...

0 Kudos

2,536 Views
daiane_angolini
NXP Employee
NXP Employee

Can you, please, see if this DOC can help you?

https://community.freescale.com/docs/DOC-102133

2,536 Views
nathanmak
Contributor II

Hi Daiane,

The link you posted is broken. I get an unauthorized access page.

Could you update me if IPU1/CSI1 support has been rolled in as Omar has requested and attempted to do with his own patch?

I am trying to do something similar (a second adv7180 on CSI1 feeding IPU2) and this is all very helpful. Actually I think Omar meant IPU2/CSI1 but I could be wrong.

Thanks

0 Kudos

2,536 Views
daiane_angolini
NXP Employee
NXP Employee

Hi nathanmak, I got unauthorized access page as well. I think the DOC was removed.

Unfortunately, I don't have how to provide it again.

0 Kudos

2,536 Views
Selea
Senior Contributor I

this is my patch file adapted to the boundary kernel 3.10.17.1.0.1_ga kernel tree to use both parallel, but still have problem.

0 Kudos

2,536 Views
Selea
Senior Contributor I

well... I adapted the patch to the boundary kernel, but still i cannot acquire from the IPU1/CSI1 parallel...

still working on it..

Omar

0 Kudos

2,536 Views
Selea
Senior Contributor I

Time to merge that patch in the kernel. I think several people on this community are trying to make the second parallel CSI port working.....

Great job!

Omar

2,536 Views
Selea
Senior Contributor I

Now i will test and I will let you know. But it looks good way to fix it.

Omar

0 Kudos

2,536 Views
Selea
Senior Contributor I

the patch doesn't apply directly to 3.10.17_1.0.1_ga... some minor difference... I'm trying to fix.

0 Kudos