Use FPGA as camera input - IPU CSI Parallel

[haggaytavyumi]

Hello all

We have developed our own custom  board based on the iMX6D processor.

The board include also an FPGA used for image manipulations.

The FPGA sends the output image to the iMX6D using parallel CSI.

We have build the yocto 3.14.52 from the freescale git source:

meta-fsl-bsp-release.git - Freescale i.MX Yocto BSP Release Layer

The build command was based on the SabreSD build.

After that we made some modifications to the DTS and the kernel to fit our board.

Most of the changes are working good.

Only one issue is the CSI capturing the video from the FPGA.

the pins I have used to connect the video in are :

pinctrl_ipu1_1: ipu1grp-1 {

            fsl,pins = </*parallel camera interface 1*/

                MX6QDL_PAD_CSI0_DAT4__IPU1_CSI0_DATA04        0x10

                MX6QDL_PAD_CSI0_DAT5__IPU1_CSI0_DATA05        0x10

                MX6QDL_PAD_CSI0_DAT6__IPU1_CSI0_DATA06        0x10

                MX6QDL_PAD_CSI0_DAT7__IPU1_CSI0_DATA07        0x10

                MX6QDL_PAD_CSI0_DAT8__IPU1_CSI0_DATA08        0x10

                MX6QDL_PAD_CSI0_DAT9__IPU1_CSI0_DATA09        0x10

                MX6QDL_PAD_CSI0_DAT10__IPU1_CSI0_DATA10        0x10

                MX6QDL_PAD_CSI0_DAT11__IPU1_CSI0_DATA11        0x10

                MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12        0x10

                MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13        0x10

                MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14        0x10

                MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15        0x10

                MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16        0x10

                MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17        0x10

                MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18        0x10

                MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19        0x10

                MX6QDL_PAD_CSI0_DATA_EN__IPU1_CSI0_DATA_EN    0x10

                MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC        0x10

                MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK    0x10

                MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC        0x10

                >;

        };

The format of the video is UYVY 16-bit 1280X720P.

The polarity fit to the IMX6SDLCEC document Figure 62.

We made a new driver in the linux-imx\drivers\media\platform\mxc\capture\ folder based on the OV5642 driver.

The driver is loading and a device have been crated in the /dev directory /dev/video0.

I have printed out the registers values:

CSI_SENS_FRM_SIZE = 0x2CF04FF

CSI_CCIR_CODE_1 = 0x0

CSI_CCIR_CODE_2 = 0x0

CSI_CCIR_CODE_3 = 0x0

CSI_SENS_CONF = 0x0000CB00

CSI_ACT_FRM_SIZE = 0x2CF04FF

I have tried to use unit_tests tools to find out if the capture source is working.

I have used the command :

./mxc_v4l2_capture.out -iw 1280 -ih 720 -ow 640 -oh 480 -m 0 -r 0 -c 50 -f UYVY -fr 60 -d /dev/video0 test.yuv

As a result I have this error :"ERROR: v4l2 capture: VIDIOC_QBUF: buffer already queued" "VIDIOC_QBUF failed"

The size of the output file is 600kB.

After that I used the ./mxc_v4l2_output.out -iw 1280 -ih 720 -ow 640 -oh 480 -r 0 -f UYVY -fr 60 test.yuv command to view the file.

As a result to that I have the error :

"start time = 1454478394 s, 817478 us

v4l2_output: end of input file, g_frame_size=1843200, err = 614400

v4l2_output: no display because v4l need at least 1 frames

total time for 0 frames = 15315 us =  0 fps

"

Is there a way someone can help me get my hands on that problem?

[sylvainlehenaff]

Hi,

I am currently trying to do similar work with a IMX6 and an FPGA sending video data. The fpga is only sending data without any control. How did you make the system "think" that the fpga/camera was connected up and running ? The FPGA will not answer any probing request as it is basic ?

I understand that i will have to make some modification based on the OV54xx driver but I don t understand how to bypass the detection part.

Do you have any advice ?

regards,

sylvain

[haggaytavyumi]

Hi Sylvain

I hope it is not too late.

You have described the process very well.I have took the OV54XX driver and rewrote it without the I2C elements.I Can't publish my entire work here.

I will be happy to help you with any specific question.

[sylvainlehenaff]

Hi,

I was off for summer break, but I am still working on that feature. I manage to get a code (based on the IMX Unit test for the camera) that program the various parameter (inc. iomux) and make some kink of capture.

When it is connected on the FPGA, nothing comes out at the "IPU interface" despite that I am quite sure that the data are coming out of the FPGA, I should have some configuration issues (maybe on the DTB part). I am getting soon a camera module to try to isolate the problem.

regards 

[haggaytavyumi]

Hi Sylvain

What is the BSP version you use?

Have you changed the machine name in the DTS?

Because if you did my solution will be super relevant.

good lack.

[igorpadykov]

Hi haggay

Table 66. Camera Input Signal Cross Reference, Format, and Bits Per Cycle

datasheet shows supported configurations, seems this is supported as a “generic-data”

input and some examples can be found on

https://community.freescale.com/thread/383366

https://community.freescale.com/message/331888

Best regards

igor

-----------------------------------------------------------------------------------------------------------------------

Note: If this post answers your question, please click the Correct Answer button. Thank you!

-----------------------------------------------------------------------------------------------------------------------

[haggaytavyumi]

Hello Igor

Thank you for the quick replay.

You are right I use the “generic-data” mode.

After reading what Marek Vasut did, I tried to do the same.

I reconfigured my FPGA to "none gated" instead of "gated" "generic-data" mode.

So now the my printed registers look like:

CSI_SENS_FRM_SIZE = 0x2ED04FF

CSI_CCIR_CODE_1 = 0x0

CSI_CCIR_CODE_2 = 0x0

CSI_CCIR_CODE_3 = 0x0

CSI_SENS_CONF = 0xCB10

And I have printed out the debug messages in function _ipu_ch_param_dump() :

imx-ipuv3 2400000.ipu: ch 20 word 0 - 00000000 00000000 00000000 E0001800 00077C4F

imx-ipuv3 2400000.ipu: ch 20 word 1 - 08C20000 01184000 2143C000 00013FC0 00000000

imx-ipuv3 2400000.ipu: PFS 0xa,

imx-ipuv3 2400000.ipu: BPP 0x3,

imx-ipuv3 2400000.ipu: NPB 0xf

imx-ipuv3 2400000.ipu: FW 639,

imx-ipuv3 2400000.ipu: FH 479,

imx-ipuv3 2400000.ipu: EBA0 0x46100000

imx-ipuv3 2400000.ipu: EBA1 0x46100000

imx-ipuv3 2400000.ipu: Stride 1279

imx-ipuv3 2400000.ipu: scan_order 0

imx-ipuv3 2400000.ipu: uv_stride 0

imx-ipuv3 2400000.ipu: u_offset 0x0

imx-ipuv3 2400000.ipu: v_offset 0x0

imx-ipuv3 2400000.ipu: Width0 0+1,

imx-ipuv3 2400000.ipu: Width1 0+1,

imx-ipuv3 2400000.ipu: Width2 0+1,

imx-ipuv3 2400000.ipu: Width3 0+1,

imx-ipuv3 2400000.ipu: Offset0 0,

imx-ipuv3 2400000.ipu: Offset1 0,

imx-ipuv3 2400000.ipu: Offset2 0,

imx-ipuv3 2400000.ipu: Offset3 0

For now I have the same error :

"ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

VIDIOC_DQBUF failed.

ERROR: v4l2 capture: VIDIOC_QBUF: buffer already queued

VIDIOC_QBUF failed"

[haggaytavyumi · Accepted solution]

Hello All

Thanks for the "help".

After real digging in to the registers of the IPU. I found out that the IOMUXC_GPR1 register is modified by mach file mach-imx6q.c line 347.

The modification depend on the boards name from the device tree DTS file.

My customized board is not on the list(that contain only the SabreSD and the SabreAUTO).

In bit number 19 and 20 of the IOMUXC_GPR1 register defined if the CSI input is from the parallel pins or from the MIPI pins.

I added to my camera driver check of the  IOMUXC_GPR1 register and fix the values base on the device tree values.

This way the muxing is based on the Device tree and the driver and not on a specific board.

Here is the code added:

int retval,ipu_id;

static void *ADDR;

volatile u32 *reg;

retval = of_property_read_u32(dev->of_node, "ipu_id", &ipu_id);

    if (retval) {

        dev_err(&pdev->dev, "ipu_id missing or invalid\n");

        return retval;

    }

    else{

        ADDR = ioremap(0x020E0000, 32);

        reg= (u32 *)(ADDR+4);

        printk("%s %s %s: IOMUXC_GPR1 = 0x%X ",MAGENTA,__func__,CLEAR_COLOR,readl(reg));

        if(!ipu_id){

            *reg= *reg | 1<<19;

        }

        else{

            *reg= *reg | 1<<20;

        }

    }

Now I get my frames.