Custom sensor V4L2 driver for HummingBoard-i2eX (i.MX6 Dual)-Issue

cancel
Showing results for 
Search instead for 
Did you mean: 

Custom sensor V4L2 driver for HummingBoard-i2eX (i.MX6 Dual)-Issue

Jump to solution
1,949 Views
Contributor II

Hello All,

I am developing V4L2 image sensor driver for a custom sensor.

Platform details are:

Board : HummingBoard-i2eX (i.MX6 Dual)

OS      : Ubuntu, Linux Kernel 3.0.35

Sensor details:

1. Supported MIPI lanes : 2 Lane/4 Lane (configured to use 2 lane mode)

2. Supported MIPI data format : RAW 10 /RAW8 (configured to use RAW10)

3. Virtual channel : configured to 0

4. Clock Frequency : 24MHz (This is actually sensor clock, MIPI clock is 114 MHz)

5. Resolution : 1280 x 720

I am getting error "ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0" for  VIDIOC_DQBUF IOCTL.

To begin my trouble shooting, I would like to get my below board configuration file modifications reviewed:

//Sensor details

static struct fsl_mxc_camera_platform_data mipi_csi2_data = {

  .mclk = 24000000, // Is this OK ? Or should I put the MIPI clock(114 MHz) here?

  .mclk_source = 0, // Does this indicate HB MIPI clock source is sensor?

  .csi = 0, // Does this indicate camera sensor is using IPU #1/CSI0 ?

  .io_init = mx6q_csi0_io_init, // I use this function to Enable mipi to IPU1 CSI0 (mxc_iomux_set_gpr_register(1, 19, 1, 0))

};

Since this structure definition doesn't have any documentation, I did not understand the meaning of these parameters. Please explain me these parameters if anybody is aware of.

//Details of MIPI CSI2 interface

static struct mipi_csi2_platform_data mipi_csi2_pdata = {

       .ipu_id   = 0,

       .csi_id = 0,

         .v_channel = 0,

       .lanes =2,

         .dphy_clk = "mipi_pllref_clk",

         .pixel_clk = "emi_clk",

};

/*Details of capture device*/

static struct fsl_mxc_capture_platform_data capture_data[] = {

   {

        .csi = 0,

        .ipu = 0,

        .mclk_source = 0,

        .is_mipi = 1,

    },  

};

Actually, I took these values from another sensor patch, i dont really know what these parameters are

Now, assuming that my board configuration is ok, I went ahead with driver debugging. Below are the observations:

1. I2C communication is ok

2. MIPI_CSI_PHY_STATE value is : 630,sometimes 330,300 also (Is this ok or is it indicating any clock related issue?)

3. MIPI_CSI_ERR1/2 both return 0

Any hints to my issue? Can this be related to the format my sensor support (RAW 10)? Can someone direct me how to proceed with debugging?

Also, in mxc_mipi driver, I saw apis to read MIPI_CSI_PHY_STATE and  MIPI_CSI_ERR1/2 registers,however, I didnt find any apis to read other registers

(MIPI_CSI_N_LANES ~ MIPI_CSI_DATA_IDS_2). Can someone guide me on how to read the other registers (also, IPU registers) ?

Labels (2)
0 Kudos
1 Solution
23 Views
Contributor II

Update:

1. I got the issue resolved by switching to latest kernel (3.14 FSL) and adding some basic RAW10 support in BSP.

2. Also, I had to set p->if_type as V4L2_IF_TYPE_BT656 (V4L2_IF_TYPE_BT656_PROGRESSIVE did not work,don't know the reason ) in ioctl_g_ifparm of the sensor driver.

View solution in original post

0 Kudos
10 Replies
24 Views
Contributor II

Update:

1. I got the issue resolved by switching to latest kernel (3.14 FSL) and adding some basic RAW10 support in BSP.

2. Also, I had to set p->if_type as V4L2_IF_TYPE_BT656 (V4L2_IF_TYPE_BT656_PROGRESSIVE did not work,don't know the reason ) in ioctl_g_ifparm of the sensor driver.

View solution in original post

0 Kudos
23 Views
Contributor II

Hi Rejeesh Babu,

1. I got the issue resolved by switching to latest kernel (3.14 FSL) and adding some basic RAW10 support in BSP.

 Raw10 data can be displayed on the imx6?(not use ISP/DSP)

0 Kudos
23 Views
Contributor III

Hi @ken meng,

   I have managed to store frames from ov9724 camera to memory using the following command.

./mxc_v4l2_capture.out -iw 1280 -ih 720 -ow 1280 -oh 720 -m 1 -i 1 -r 0 -c 10 -fr 30 -d /dev/video1 /mnt/4.data

I am also attaching the camera data file here. How do I go about converting it to RGB? It is in the IPU Generic format. 

Actually, the camera streams it in RAW-10 format, but I configured the IPU for V4L2_PIX_FMT_SBGGR10 format, which the IPU detects as IPU_PIX_FMT_GENERIC format. I am unsure as to how to go about processing the raw images. How to verify if the data saved in the file is in the correct format? 

I am attaching the hex dump of the data from the frame file. Seems that every 3rd and fourth lines (one line = 16byte) are the same. Does this look like a valid raw data image? 

00000000 89 24 07 1f c7 1f 88 21 c7 1e c8 22 48 20 c7 1f       |.$.....!..."H ..|
00000010 c7 1e 4a 2b 48 22 09 26 48 20 49 26 c8 21 09 26    |..J+H".&H I&.!.&|
00000020 d4 50 0b 2f 15 54 cd 34 91 47 8b 2f 11 46 4c 33       |.P./.T.4.G./.FL3|
*
00000040 c7 1d 4a 29 48 22 49 27 48 22 08 20 88 23 88 21    |..J)H"I'H". .#.!|
00000050 08 21 89 26 c8 21 09 25 48 20 08 21 86 1b c8 23    |.!.&.!.%H .!...#|
00000060 13 4d 0e 38 11 46 4b 2d 51 47 4b 2c d0 41 4e 39    |.M.8.FK-QGK,.AN9|
*
00000080 48 22 09 27 87 1d c8 22 c7 1c 49 24 87 1d 08 21    |H".'..."..I$...!|
00000090 88 22 88 20 47 1f c8 21 c6 19 48 22 87 1e 87 1e     |.". G..!..H"....|
000000a0 10 41 4a 2a 11 44 cb 2f 11 46 0c 30 0f 3d 8c 30      |.AJ*.D./.F.0.=.0|
*
000000c0 c6 1a 87 1e c7 1c 46 1a 87 1f 89 24 07 1e c8 20     |......F....$... |
000000d0 86 1b c7 1d c6 1a 08 23 47 1e 07 1d 47 1c 88 23    |.......#G...G..#|
000000e0 4f 3f 4c 32 14 50 8b 2f 91 47 8e 39 d1 46 cd 34       |O?L2.P./.G.9.F.4|
*
00000100 c9 25 09 25 08 22 4b 2c 08 21 0a 2a c9 27 8b 2e    |.%.%."K,.!.*.'..|
00000110 49 26 cc 33 8a 2a 49 27 49 27 4a 29 c8 22 ca 29    |I&.3.*I'I'J).".)|
00000120 d1 46 cd 35 94 51 8e 39 54 52 8c 33 51 46 4d 37    |.F.5.Q.9TR.3QFM7|
*
00000140 07 1e 4a 2a 4a 29 89 25 09 26 8b 2d 49 25 ca 2a    |..J*J).%.&.-I%.*|
00000150 4a 28 8a 29 0a 28 49 26 8a 28 8b 2c 0a 29 49 27    |J(.).(I&.(.,.)I'|
00000160 56 59 8e 38 94 51 0d 37 53 4f 8f 3c 14 51 4e 38      |VY.8.Q.7SO.<.QN8|

0 Kudos
23 Views
Contributor II

Hi All,


Assuming that my problem is lack of RAW format support for freescale V4L2 framework, I went ahead and applied the patch mentioned in https://community.freescale.com/thread/307065.

Now my configuration regsiters look better, though I am getting the same error. i have enabled more logging and below is the output. Any tip from the log?

In MVC: mxc_v4l_open

   device name is Mxc Camera

End of mxc_v4l_open: v2f pix widthxheight 1280 x 720

End of mxc_v4l_open: crop_bounds widthxheight 1280 x 720

End of mxc_v4l_open: crop_defrect widthxheight 1280 x 720

End of mxc_v4l_open: crop_current widthxheight 1280 x 720

On Open: Input to ipu size is 1280 x 720

Pixel format is : IPU_PIX_FMT_GENERIC,0x32555049

Data written to CSI_SENS_CONF: 9b20

Sensor clock mode is: IPU_CSI_CLK_MODE_CCIR656_PROGRESSIVE

imx-ipuv3 imx-ipuv3.0: CSI_SENS_CONF = 0x00009B20

imx-ipuv3 imx-ipuv3.0: CSI_ACT_FRM_SIZE = 0x02CF04FF

mipi_reg during device init=330

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 80685600

   case VIDIOC_QUERYCAP

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c563a

   case VIDIOC_CROPCAP

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 4014563c

   case VIDIOC_S_CROP

   Cropping Input to ipu size 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0cc5604

   case VIDIOC_G_FMT

In MVC: mxc_v4l2_g_fmt type=1

   type is V4L2_BUF_TYPE_VIDEO_CAPTURE

End of mxc_v4l2_g_fmt: v2f pix widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_bounds widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_current widthxheight 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0145608

   case VIDIOC_REQBUFS

In MVC:mxc_streamoff

MVC: In mxc_free_frame_buf

mxc_free_frame_buf: allocated buffer count = 0

In MVC:mxc_allocate_frame_buf - size=921600

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

   case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

   pgoff=0x18900, start=0x2aca3000, end=0x2ad84000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

   case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

   pgoff=0x18e00, start=0x2add6000, end=0x2aeb7000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

   case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

   pgoff=0x18f00, start=0x2aed5000, end=0x2afb6000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

   case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

   pgoff=0x19000, start=0x2b061000, end=0x2b142000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

   case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

   case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

   case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

   case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 40045612

   case VIDIOC_STREAMON

In MVC:mxc_streamon with pixel format = 0x30314247

IPU:In csi_enc_enabling_tasks

In csi_enc_setup with format = 0x30314247, ipu = 0, csi = 0

cam->csi =0

setting up ipu csi enc with mipi vc = 0

imx-ipuv3 imx-ipuv3.0: init channel = 15

Inside _ipu_csi_init

Destination is CSI_DATA_DEST_IDMAC

setting up ipu 0 from csi_enc_setup with pixel format = 0x30555049

imx-ipuv3 imx-ipuv3.0: initializing idma ch 0 @ c08c0000

imx-ipuv3 imx-ipuv3.0: ch 0 word 0 - 00000000 00000000 00000000 E0002800 000B3C9F

imx-ipuv3 imx-ipuv3.0: ch 0 word 1 - 03220000 00644000 00C7C000 00013FC0 00000000

imx-ipuv3 imx-ipuv3.0: PFS 0x6,

imx-ipuv3 imx-ipuv3.0: BPP 0x5,

imx-ipuv3 imx-ipuv3.0: NPB 0x1f

imx-ipuv3 imx-ipuv3.0: FW 1279,

imx-ipuv3 imx-ipuv3.0: FH 719,

imx-ipuv3 imx-ipuv3.0: EBA0 0x19100000

imx-ipuv3 imx-ipuv3.0: EBA1 0x19100000

imx-ipuv3 imx-ipuv3.0: Stride 1279

imx-ipuv3 imx-ipuv3.0: scan_order 0

imx-ipuv3 imx-ipuv3.0: uv_stride 0

imx-ipuv3 imx-ipuv3.0: u_offset 0x0

imx-ipuv3 imx-ipuv3.0: v_offset 0x0

imx-ipuv3 imx-ipuv3.0: Width0 0+1,

imx-ipuv3 imx-ipuv3.0: Width1 0+1,

imx-ipuv3 imx-ipuv3.0: Width2 0+1,

imx-ipuv3 imx-ipuv3.0: Width3 0+1,

imx-ipuv3 imx-ipuv3.0: Offset0 0,

imx-ipuv3 imx-ipuv3.0: Offset1 0,

imx-ipuv3 imx-ipuv3.0: Offset2 0,

imx-ipuv3 imx-ipuv3.0: Offset3 0

In MVC:mxc_poll

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445611

   case VIDIOC_DQBUF

In MVC:mxc_v4l_dqueue

In MVC:Dump Registers

In MVC:Inside Dump Registers

MIPI_CSI_N_LANES                : 1

MIPI_CSI_PHY_SHUTDOWNZ  : 1

MIPI_CSI_DPHY_RSTZ              : 1

MIPI_CSI_CSI2_RESETN    : 1

MIPI_CSI_PHY_STATE              : 330

MIPI_CSI_DATA_IDS_1             : 0

MIPI_CSI_DATA_IDS_2             : 0

IPU1_CONF                               : 10000761

IPUx_INT_STAT_1                 : 800000

IPUx_CSI0_DI                    : ffffff2b

IPU1_CSI0_SENS_CONF             : 4009b20

IPU1_CSI0_SENS_FRM_SIZE : 2cf04ff

IPU1_CSI0_ACT_FRM_SIZE          : 2cf04ff

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

In MVC:mxc_v4l_close

In MVC:mxc_streamoff

imx-ipuv3 imx-ipuv3.0: CSI stop timeout - 5 * 10ms

In MVC:mxc_free_frames

[520] : drivers/media/video/mxc/capture/imx219.c : ioctl_s_power() ENTER

[617] : drivers/media/video/mxc/capture/imx219.c : ioctl_s_power() EXIT

mxc_v4l_close: release resource

MVC: In mxc_free_frame_buf

mxc_free_frame_buf: allocated buffer count = -4

In MVC:mxc_free_frames

0 Kudos
23 Views
Contributor V

Dear Rejeesh Babu,

I'm new to this MIPI CSI interface and I have the custom camera, not able to get the a frame while running v4l2 application.

Can you please provide any suggestions on my issue ?

https://community.nxp.com/message/843732?commentID=843732#comment-843732 

Much appreciated on your help.

0 Kudos
23 Views
Contributor V

I have solved the problem.
https://community.nxp.com/thread/437356

0 Kudos
23 Views
Contributor II

Hi All,

I managed to read few registers related to IPU and MIPI. One thing I quickly noticed is that, value of  IPU1_CONF suggest that CSI0 if IPU1 is disabled (which is what I am trying to use!). Can somebdy let me know how to enable CSI0 if IPU1 (just writing to this bit from sensor driver is OK?).

Also, it would be great if  someone could check the values of other registers as well and make more suggestions.

MIPI_CSI_N_LANES : 0x 1

MIPI_CSI_PHY_SHUTDOWNZ : 0x 1

MIPI_CSI_DPHY_RSTZ : 0x 1

MIPI_CSI_CSI2_RESETN : 0x 1

MIPI_CSI_PHY_STATE : 0x 630

MIPI_CSI_DATA_IDS_1 : 0x 0

MIPI_CSI_DATA_IDS_2 : 0x 0

IPU1_CONF : 0x660

IPUx_INT_STAT_1 : 0x800000

IPUx_CSI0_DI : 0xffffffff

IPU1_CSI0_SENS_CONF : 0x 0

IPU1_CSI0_SENS_FRM_SIZE : 0x 0

IPU1_CSI0_ACT_FRM_SIZE : 0x1df027f

0 Kudos
23 Views
Senior Contributor II

Hi Rejeesh Babu,

In this file drivers/mxc/mipi/mxc_mipi_csi2.c

do the below change.

mipi_csi2_write(info, 0x00000014, MIPI_CSI2_PHY_TST_CTRL1);

mipi_csi2_write(info, 0x00000002, CSI2_PHY_TST_CTRL1); /* 110-125MHz */

Regards,

Aravinth

0 Kudos
23 Views
Contributor II

It did not help Aravind; btw how did you get this value? Also how do I ensure that my PIX_CLK (and any other clock related to capturing) is correct?

0 Kudos
23 Views
Senior Contributor II

Hi Rejeesh,

From this  ( Debug steps for customer MIPI sensor.docx ) freescale debug document they mentioned that PLL CLK(Differential clock)range.

0 Kudos