Custom sensor V4L2 driver for HummingBoard-i2eX (i.MX6 Dual)-Issue

rejeeshbabu · ‎02-04-2016

Hello All,

I am developing V4L2 image sensor driver for a custom sensor.

Platform details are:

Board : HummingBoard-i2eX (i.MX6 Dual)

OS : Ubuntu, Linux Kernel 3.0.35

Sensor details:

1. Supported MIPI lanes : 2 Lane/4 Lane (configured to use 2 lane mode)

2. Supported MIPI data format : RAW 10 /RAW8 (configured to use RAW10)

3. Virtual channel : configured to 0

4. Clock Frequency : 24MHz (This is actually sensor clock, MIPI clock is 114 MHz)

5. Resolution : 1280 x 720

I am getting error "ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0" for VIDIOC_DQBUF IOCTL.

To begin my trouble shooting, I would like to get my below board configuration file modifications reviewed:

//Sensor details

static struct fsl_mxc_camera_platform_data mipi_csi2_data = {

.mclk = 24000000, // Is this OK ? Or should I put the MIPI clock(114 MHz) here?

.mclk_source = 0, // Does this indicate HB MIPI clock source is sensor?

.csi = 0, // Does this indicate camera sensor is using IPU #1/CSI0 ?

.io_init = mx6q_csi0_io_init, // I use this function to Enable mipi to IPU1 CSI0 (mxc_iomux_set_gpr_register(1, 19, 1, 0))

};

Since this structure definition doesn't have any documentation, I did not understand the meaning of these parameters. Please explain me these parameters if anybody is aware of.

//Details of MIPI CSI2 interface

static struct mipi_csi2_platform_data mipi_csi2_pdata = {

.ipu_id = 0,

.csi_id = 0,

.v_channel = 0,

.lanes =2,

.dphy_clk = "mipi_pllref_clk",

.pixel_clk = "emi_clk",

};

/*Details of capture device*/

static struct fsl_mxc_capture_platform_data capture_data[] = {

{

.csi = 0,

.ipu = 0,

.mclk_source = 0,

.is_mipi = 1,

},

};

Actually, I took these values from another sensor patch, i dont really know what these parameters are

Now, assuming that my board configuration is ok, I went ahead with driver debugging. Below are the observations:

1. I2C communication is ok

2. MIPI_CSI_PHY_STATE value is : 630,sometimes 330,300 also (Is this ok or is it indicating any clock related issue?)

3. MIPI_CSI_ERR1/2 both return 0

Any hints to my issue? Can this be related to the format my sensor support (RAW 10)? Can someone direct me how to proceed with debugging?

Also, in mxc_mipi driver, I saw apis to read MIPI_CSI_PHY_STATE and MIPI_CSI_ERR1/2 registers,however, I didnt find any apis to read other registers

(MIPI_CSI_N_LANES ~ MIPI_CSI_DATA_IDS_2). Can someone guide me on how to read the other registers (also, IPU registers) ?

rejeeshbabu · ‎03-08-2016

Update:

1. I got the issue resolved by switching to latest kernel (3.14 FSL) and adding some basic RAW10 support in BSP.

2. Also, I had to set p->if_type as V4L2_IF_TYPE_BT656 (V4L2_IF_TYPE_BT656_PROGRESSIVE did not work,don't know the reason ) in ioctl_g_ifparm of the sensor driver.

View solution in original post

rejeeshbabu · ‎03-08-2016

Update:

1. I got the issue resolved by switching to latest kernel (3.14 FSL) and adding some basic RAW10 support in BSP.

2. Also, I had to set p->if_type as V4L2_IF_TYPE_BT656 (V4L2_IF_TYPE_BT656_PROGRESSIVE did not work,don't know the reason ) in ioctl_g_ifparm of the sensor driver.

kenmeng · ‎11-13-2016

Hi Rejeesh Babu,

1. I got the issue resolved by switching to latest kernel (3.14 FSL) and adding some basic RAW10 support in BSP.

Raw10 data can be displayed on the imx6?（not use ISP/DSP）

navinars · ‎01-17-2017

Hi @ken meng,

I have managed to store frames from ov9724 camera to memory using the following command.

./mxc_v4l2_capture.out -iw 1280 -ih 720 -ow 1280 -oh 720 -m 1 -i 1 -r 0 -c 10 -fr 30 -d /dev/video1 /mnt/4.data

I am also attaching the camera data file here. How do I go about converting it to RGB? It is in the IPU Generic format.

Actually, the camera streams it in RAW-10 format, but I configured the IPU for V4L2_PIX_FMT_SBGGR10 format, which the IPU detects as IPU_PIX_FMT_GENERIC format. I am unsure as to how to go about processing the raw images. How to verify if the data saved in the file is in the correct format?

I am attaching the hex dump of the data from the frame file. Seems that every 3rd and fourth lines (one line = 16byte) are the same. Does this look like a valid raw data image?

rejeeshbabu · ‎02-12-2016

Hi All,

Assuming that my problem is lack of RAW format support for freescale V4L2 framework, I went ahead and applied the patch mentioned in https://community.freescale.com/thread/307065.

Now my configuration regsiters look better, though I am getting the same error. i have enabled more logging and below is the output. Any tip from the log?

In MVC: mxc_v4l_open

device name is Mxc Camera

End of mxc_v4l_open: v2f pix widthxheight 1280 x 720

End of mxc_v4l_open: crop_bounds widthxheight 1280 x 720

End of mxc_v4l_open: crop_defrect widthxheight 1280 x 720

End of mxc_v4l_open: crop_current widthxheight 1280 x 720

On Open: Input to ipu size is 1280 x 720

Pixel format is : IPU_PIX_FMT_GENERIC,0x32555049

Data written to CSI_SENS_CONF: 9b20

Sensor clock mode is: IPU_CSI_CLK_MODE_CCIR656_PROGRESSIVE

imx-ipuv3 imx-ipuv3.0: CSI_SENS_CONF = 0x00009B20

imx-ipuv3 imx-ipuv3.0: CSI_ACT_FRM_SIZE = 0x02CF04FF

mipi_reg during device init=330

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 80685600

case VIDIOC_QUERYCAP

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c02c563a

case VIDIOC_CROPCAP

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 4014563c

case VIDIOC_S_CROP

Cropping Input to ipu size 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0cc5604

case VIDIOC_G_FMT

In MVC: mxc_v4l2_g_fmt type=1

type is V4L2_BUF_TYPE_VIDEO_CAPTURE

End of mxc_v4l2_g_fmt: v2f pix widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_bounds widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1280 x 720

End of mxc_v4l2_g_fmt: crop_current widthxheight 1280 x 720

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0145608

case VIDIOC_REQBUFS

In MVC:mxc_streamoff

MVC: In mxc_free_frame_buf

mxc_free_frame_buf: allocated buffer count = 0

In MVC:mxc_allocate_frame_buf - size=921600

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

pgoff=0x18900, start=0x2aca3000, end=0x2ad84000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

pgoff=0x18e00, start=0x2add6000, end=0x2aeb7000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

pgoff=0x18f00, start=0x2aed5000, end=0x2afb6000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445609

case VIDIOC_QUERYBUF

In MVC:mxc_v4l2_buffer_status

In MVC:mxc_mmap

pgoff=0x19000, start=0x2b061000, end=0x2b142000

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c044560f

case VIDIOC_QBUF

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl 40045612

case VIDIOC_STREAMON

In MVC:mxc_streamon with pixel format = 0x30314247

IPU:In csi_enc_enabling_tasks

In csi_enc_setup with format = 0x30314247, ipu = 0, csi = 0

cam->csi =0

setting up ipu csi enc with mipi vc = 0

imx-ipuv3 imx-ipuv3.0: init channel = 15

Inside _ipu_csi_init

Destination is CSI_DATA_DEST_IDMAC

setting up ipu 0 from csi_enc_setup with pixel format = 0x30555049

imx-ipuv3 imx-ipuv3.0: initializing idma ch 0 @ c08c0000

imx-ipuv3 imx-ipuv3.0: ch 0 word 0 - 00000000 00000000 00000000 E0002800 000B3C9F

imx-ipuv3 imx-ipuv3.0: ch 0 word 1 - 03220000 00644000 00C7C000 00013FC0 00000000

imx-ipuv3 imx-ipuv3.0: PFS 0x6,

imx-ipuv3 imx-ipuv3.0: BPP 0x5,

imx-ipuv3 imx-ipuv3.0: NPB 0x1f

imx-ipuv3 imx-ipuv3.0: FW 1279,

imx-ipuv3 imx-ipuv3.0: FH 719,

imx-ipuv3 imx-ipuv3.0: EBA0 0x19100000

imx-ipuv3 imx-ipuv3.0: EBA1 0x19100000

imx-ipuv3 imx-ipuv3.0: Stride 1279

imx-ipuv3 imx-ipuv3.0: scan_order 0

imx-ipuv3 imx-ipuv3.0: uv_stride 0

imx-ipuv3 imx-ipuv3.0: u_offset 0x0

imx-ipuv3 imx-ipuv3.0: v_offset 0x0

imx-ipuv3 imx-ipuv3.0: Width0 0+1,

imx-ipuv3 imx-ipuv3.0: Width1 0+1,

imx-ipuv3 imx-ipuv3.0: Width2 0+1,

imx-ipuv3 imx-ipuv3.0: Width3 0+1,

imx-ipuv3 imx-ipuv3.0: Offset0 0,

imx-ipuv3 imx-ipuv3.0: Offset1 0,

imx-ipuv3 imx-ipuv3.0: Offset2 0,

imx-ipuv3 imx-ipuv3.0: Offset3 0

In MVC:mxc_poll

In MVC:mxc_v4l_ioctl

In MVC: mxc_v4l_do_ioctl c0445611

case VIDIOC_DQBUF

In MVC:mxc_v4l_dqueue

In MVC:Dump Registers

In MVC:Inside Dump Registers

MIPI_CSI_N_LANES : 1

MIPI_CSI_PHY_SHUTDOWNZ : 1

MIPI_CSI_DPHY_RSTZ : 1

MIPI_CSI_CSI2_RESETN : 1

MIPI_CSI_PHY_STATE : 330

MIPI_CSI_DATA_IDS_1 : 0

MIPI_CSI_DATA_IDS_2 : 0

IPU1_CONF : 10000761

IPUx_INT_STAT_1 : 800000

IPUx_CSI0_DI : ffffff2b

IPU1_CSI0_SENS_CONF : 4009b20

IPU1_CSI0_SENS_FRM_SIZE : 2cf04ff

IPU1_CSI0_ACT_FRM_SIZE : 2cf04ff

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

In MVC:mxc_v4l_close

In MVC:mxc_streamoff

imx-ipuv3 imx-ipuv3.0: CSI stop timeout - 5 * 10ms

In MVC:mxc_free_frames

[520] : drivers/media/video/mxc/capture/imx219.c : ioctl_s_power() ENTER

[617] : drivers/media/video/mxc/capture/imx219.c : ioctl_s_power() EXIT

mxc_v4l_close: release resource

MVC: In mxc_free_frame_buf

mxc_free_frame_buf: allocated buffer count = -4

In MVC:mxc_free_frames

titusstalin · ‎10-22-2016

Dear Rejeesh Babu,

I'm new to this MIPI CSI interface and I have the custom camera, not able to get the a frame while running v4l2 application.

Can you please provide any suggestions on my issue ?

https://community.nxp.com/message/843732?commentID=843732#comment-843732

Much appreciated on your help.

titusstalin · ‎11-13-2016

I have solved the problem.
https://community.nxp.com/thread/437356

rejeeshbabu · ‎02-04-2016

Hi All,

I managed to read few registers related to IPU and MIPI. One thing I quickly noticed is that, value of IPU1_CONF suggest that CSI0 if IPU1 is disabled (which is what I am trying to use!). Can somebdy let me know how to enable CSI0 if IPU1 (just writing to this bit from sensor driver is OK?).

Also, it would be great if someone could check the values of other registers as well and make more suggestions.

MIPI_CSI_N_LANES : 0x 1

MIPI_CSI_PHY_SHUTDOWNZ : 0x 1

MIPI_CSI_DPHY_RSTZ : 0x 1

MIPI_CSI_CSI2_RESETN : 0x 1

MIPI_CSI_PHY_STATE : 0x 630

MIPI_CSI_DATA_IDS_1 : 0x 0

MIPI_CSI_DATA_IDS_2 : 0x 0

IPU1_CONF : 0x660

IPUx_INT_STAT_1 : 0x800000

IPUx_CSI0_DI : 0xffffffff

IPU1_CSI0_SENS_CONF : 0x 0

IPU1_CSI0_SENS_FRM_SIZE : 0x 0

IPU1_CSI0_ACT_FRM_SIZE : 0x1df027f

aravinthkumarja · ‎02-11-2016

Hi Rejeesh Babu,

In this file drivers/mxc/mipi/mxc_mipi_csi2.c

do the below change.

mipi_csi2_write(info, 0x00000014, MIPI_CSI2_PHY_TST_CTRL1);

mipi_csi2_write(info, 0x00000002, CSI2_PHY_TST_CTRL1); /* 110-125MHz */

Regards,

Aravinth

rejeeshbabu · ‎02-12-2016

It did not help Aravind; btw how did you get this value? Also how do I ensure that my PIX_CLK (and any other clock related to capturing) is correct?

aravinthkumarja · ‎02-12-2016

Hi Rejeesh,

From this ( Debug steps for customer MIPI sensor.docx ) freescale debug document they mentioned that PLL CLK(Differential clock)range.

Custom sensor V4L2 driver for HummingBoard-i2eX (i.MX6 Dual)-Issue

Custom sensor V4L2 driver for HummingBoard-i2eX (i.MX6 Dual)-Issue

i.MX6Dual

Linux