Image sensor almost working on i.MX53 CSI, but getting every even byte twice?

[sidebranch]

Hello,

we attached a FPGA to the 8-bit CSI bus on the IMX53QSB. It generates proper v/hsync/pixclk for the "gated mode".

The FPGA sends incremental byte values on the bus, 1920 x 3 (RGB) x 1080 bytes per frame.

Using the unit test "v4l2_mxc_capture.out" we capture the data in BGR24 mode using the CSI->MEM mode (-i 1).

In the capture output, we expect

00 01 02 03 04 05 06 07 08 09 etc.

but see:

00 00 02 02 04 04 06 06 08 08 etc.

Would be great if someone can provide insight on what could be causing this.

Regards, Leon.

[sidebranch]

Philip,

For your question: good catch, I have modified the mxc_v4l2_capture.c driver source code, because I think this if statement is incorrect for my case:

    /* This may not work on other platforms. Check when adding a new one.*/
    pr_debug("   clock_curr=mclk=%d\n", ifparm.u.bt656.clock_curr);
    if (ifparm.u.bt656.clock_curr == 0) {
        csi_param.clk_mode = IPU_CSI_CLK_MODE_CCIR656_INTERLACED

This construction does not make sense to me, and the comment is vague about it. My sensor (FPGA) provides all signals to the CSI interface, even the PIXCLK.  I forced to GATED mode afterwards.

I already found the problem, this seems a second case of bad documentation, very frustrating. The i.MX53 does only seem to care about the HSYNC edge, and expects a full line of contiguous data to arrive on each clock afterwards. This is contrast with the datasheet/RM which says that the HSYNC is ANDed with PIXCLK to derive the (internal) latch signal upon which valid video pixels are clocked in.

This can be verified by making the HSYNC pulse too short. You will notice that the i.MX53 will continue to capture the data after the HSYNC goes low.

[Philip_FSL]

Yes, that should not be the case.  You might want to double check the pin mux for the HSYNC to make sure you're getting the signal properly routed to the CSI.

What do these traces mean from the log above?

[   14.871003]    IPU_CSI_CLK_MODE_CCIR656_INTERLACED
[   14.875793]    IPU_CSI_CLK_MODE_GATED_CLK

[sidebranch]

Bug was that the CSI pin 12 pinmux is not initialized for board revision 4 in the loco init file.

Next problem:

The bytes on the CSI bus, during the period that HSYNC is low, are ALSO captured. This is not according to spec for the gated mode.

[sidebranch]

Found the bug in the latest BSP release, will provide a patch.

--

Leon Woestenberg

Sidebranch

[sidebranch]

Hello Philip,

this is a capture of the on-chip scope of the FPGA, when I sample at the start of active video (HSYNC holds high for the valid data during a line in gated mode):

/mxc_v4l2_capture.out -ih 1080 -ow 1920 -oh 1080 -i 1 -format BGR24 -fr 1 -c 1 out.bin

This is out.bin:

000000 00 02 02 04 04 06 06 08 08 0a 0a 0c 0c 0e 0e 10
000010 10 12 12 14 14 16 16 18 18 1a 1a 1c 1c 1e 1e 20
000020 20 22 22 24 24 26 26 28 28 2a 2a 2c 2c 2e 2e 30
000030 30 32 32 34 34 36 36 38 38 3a 3a 3c 3c 3e 3e 40
000040 40 42 42 44 44 46 46 48 48 4a 4a 4c 4c 4e 4e 50

Here is a log:

[   14.727053] In trisendo:ioctl_g_fmt_cap.
[   14.730990]    Returning size of 1920x1080
[   14.735088] End of mxc_v4l_open: v2f pix widthxheight 288 x 352
[   14.741018] End of mxc_v4l_open: crop_bounds widthxheight 1920 x 1080
[   14.747467] End of mxc_v4l_open: crop_defrect widthxheight 1920 x 1080
[   14.753996] End of mxc_v4l_open: crop_current widthxheight 1920 x 1080
[   14.760532] On Open: Input to ipu size is 1920 x 1080
[   14.765594] mxc_ipu mxc_ipu: CSI_SENS_CONF = 0x00000800
[   14.770820] mxc_ipu mxc_ipu: CSI_ACT_FRM_SIZE = 0x0437077F
[   14.776306] mxc_ipu mxc_ipu: CSI_OUT_FRM_CTRL = 0x00000000
[   14.797474] In trisendo:ioctl_init
[   14.800904] In MVC:mxc_v4l_ioctl
[   14.804140] In MVC: mxc_v4l_do_ioctl c0cc5616
[   14.808515]    case VIDIOC_S_PARM
[   14.811829] In mxc_v4l2_s_param
[   14.814970] In trisendo:ioctl_g_parm
[   14.818553]    type is V4L2_BUF_TYPE_VIDEO_CAPTURE
[   14.823344]    Current capabilities are 0
[   14.827353]    Current capturemode is 0  change to 0
[   14.832326]    Current framerate is 30  change to 1
[   14.847467] In trisendo:ioctl_s_parm
[   14.851042]    type is V4L2_BUF_TYPE_VIDEO_CAPTURE
[   14.855832] ERROR: trisendo: ioctl_s_parm: This camera does not allow frame rate changes.
[   14.864019] In trisendo:ioctl_g_ifparm
[   14.867777]    clock_curr=mclk=0
[   14.871003]    IPU_CSI_CLK_MODE_CCIR656_INTERLACED
[   14.875793]    IPU_CSI_CLK_MODE_GATED_CLK
[   14.879809]    csi_param.data_width = IPU_CSI_DATA_WIDTH_8
[   14.885295]    csi_param.Vsync_pol = 0
[   14.889052]    csi_param.Hsync_pol = 0
[   14.892800]    csi_param.ext_vsync = 0
[   14.896548] In trisendo:ioctl_g_fmt_cap.
[   14.900478]    Returning size of 1920x1080
[   14.904574]    g_fmt_cap returns widthxheight of input as 1920 x 1080
[   14.911027] mxc_ipu mxc_ipu: CSI_SENS_CONF = 0x00000800
[   14.916252] mxc_ipu mxc_ipu: CSI_ACT_FRM_SIZE = 0x0437077F
[   14.921738] mxc_ipu mxc_ipu: CSI_OUT_FRM_CTRL = 0x00000000
[   14.927239] In MVC:mxc_v4l_ioctl
[   14.930476] In MVC: mxc_v4l_do_ioctl c0045627
[   14.934833]    case VIDIOC_S_INPUT
[   14.938242] In MVC:mxc_streamoff
[   14.941470] csi_enc_select()
[   14.944353] In MVC:mxc_v4l_ioctl
[   14.947590] In MVC: mxc_v4l_do_ioctl c014563b
[   14.951947]    case VIDIOC_G_CROP
[   14.955263] In MVC:mxc_v4l_ioctl
[   14.958499] In MVC: mxc_v4l_do_ioctl 4014563c
[   14.962854]    case VIDIOC_S_CROP
[   14.966169]    Cropping Input to ipu size 1920 x 1080
[   14.971232] In MVC:mxc_v4l_ioctl
[   14.974460] In MVC: mxc_v4l_do_ioctl c0cc5605
[   14.978825]    case VIDIOC_S_FMT
[   14.982051] In MVC: mxc_v4l2_s_fmt
[   14.985451]    type=V4L2_BUF_TYPE_VIDEO_CAPTURE
[   14.989996] End of mxc_v4l2_s_fmt: v2f pix widthxheight 1920 x 1080
[   14.996264] End of mxc_v4l2_s_fmt: crop_bounds widthxheight 1920 x 1080
[   15.002887] End of mxc_v4l2_s_fmt: crop_defrect widthxheight 1920 x 1080
[   15.009597] End of mxc_v4l2_s_fmt: crop_current widthxheight 1920 x 1080
[   15.016300] In MVC:mxc_v4l_ioctl
[   15.019537] In MVC: mxc_v4l_do_ioctl c008561c
[   15.023894]    case VIDIOC_S_CTRL
[   15.027207] In MVC:mxc_v4l2_s_ctrl
[   15.030619] In MVC:mxc_v4l_ioctl
[   15.033847] In MVC: mxc_v4l_do_ioctl c0145608
[   15.038210]    case VIDIOC_REQBUFS
[   15.041611] In MVC:mxc_streamoff
[   15.044837] MVC: In mxc_free_frame_buf
[   15.048596] In MVC:mxc_allocate_frame_buf - size=6220800
[   15.100941] In MVC:mxc_v4l_ioctl
[   15.104188] In MVC: mxc_v4l_do_ioctl c0cc5604
[   15.108581]    case VIDIOC_G_FMT
[   15.111810] In MVC: mxc_v4l2_g_fmt type=1
[   15.115818]    type is V4L2_BUF_TYPE_VIDEO_CAPTURE
[   15.120620] End of mxc_v4l2_g_fmt: v2f pix widthxheight 1920 x 1080
[   15.126889] End of mxc_v4l2_g_fmt: crop_bounds widthxheight 1920 x 1080
[   15.133512] End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1920 x 1080
[   15.140246] End of mxc_v4l2_g_fmt: crop_current widthxheight 1920 x 1080
     Width = 1920     Height = 1080[   15.147517] In MVC:mxc_v4l_ioctl
     Image size = 6220800
     pixelf[   15.152942] In MVC: mxc_v4l_do_ioctl c0445609
ormat = 859981650
[   15.160080]    case VIDIOC_QUERYBUF
[   15.165016] In MVC:mxc_v4l2_buffer_status
[   15.169071] In MVC:mxc_mmap
[   15.171867]    pgoff=0x75000, start=0x2aaae000, end=0x2b09d000
[   15.194326] In MVC:mxc_v4l_ioctl
[   15.197576] In MVC: mxc_v4l_do_ioctl c0445609
[   15.201933]    case VIDIOC_QUERYBUF
[   15.205421] In MVC:mxc_v4l2_buffer_status
[   15.209450] In MVC:mxc_mmap
[   15.212245]    pgoff=0x75800, start=0x2b09d000, end=0x2b68c000
[   15.234680] In MVC:mxc_v4l_ioctl
[   15.237925] In MVC: mxc_v4l_do_ioctl c0445609
[   15.242282]    case VIDIOC_QUERYBUF
[   15.245769] In MVC:mxc_v4l2_buffer_status
[   15.249797] In MVC:mxc_mmap
[   15.252591]    pgoff=0x74000, start=0x2b68c000, end=0x2bc7b000
[   15.275049] In MVC:mxc_v4l_ioctl
[   15.278295] In MVC: mxc_v4l_do_ioctl c044560f
[   15.282653]    case VIDIOC_QBUF
[   15.285797] In MVC:mxc_v4l_ioctl
[   15.289037] In MVC: mxc_v4l_do_ioctl c044560f
[   15.293393]    case VIDIOC_QBUF
[   15.296535] In MVC:mxc_v4l_ioctl
[   15.299784] In MVC: mxc_v4l_do_ioctl c044560f
[   15.304141]    case VIDIOC_QBUF
[   15.307308] In MVC:mxc_v4l_ioctl
[   15.310560] In MVC: mxc_v4l_do_ioctl 40045612
[   15.314917]    case VIDIOC_STREAMON
[   15.318415] In MVC:mxc_streamon
[   15.321556] cam->enc_enable()
[   15.324522] IPU:In csi_enc_enabling_tasks
[   15.343520] In csi_enc_setup
[   15.357485] mxc_ipu mxc_ipu: init channel = 15
[   15.361948] initializing idma ch 0 @ e08c0000
[   15.366314] ch 0 word 0 - 00000000 00000000 00000000 E0000800 0010DCEF
[   15.372860] ch 0 word 1 - 0E900000 01D20000 00E4C000 FFF59FC0 000C0110
[   15.379397] PFS 0x7,
[   15.381495] BPP 0x1,
[   15.383775] NPB 0x13
[   15.386143] FW 1919,
[   15.388249] FH 1079,
[   15.390530] Stride 5759
[   15.393158] Width0 7+1,
[   15.744894] In csi_enc_callback()
[   15.748211] In MVC:camera_callback
[   15.751617] eba 74000000
[   16.004988] In csi_enc_callback()
[   16.008314] In MVC:camera_callback
[   16.011720] eba 74800000
buf.index 0
[   16.028205] In MVC:mxc_v4l_ioctl
[   16.031998] In MVC: mxc_v4l_do_ioctl 40045613
[   16.036355]    case VIDIOC_STREAMOFF
[   16.039945] In MVC:mxc_streamoff
[   16.237484] IPU_CONF =     0x00000760
[   16.240886] IDMAC_CONF =     0x0000002F
[   16.244460] IDMAC_CHA_EN1 =     0x00800001
[   16.248305] IDMAC_CHA_EN2 =     0x00000000
[   16.252140] IDMAC_CHA_PRI1 =     0x18800001
[   16.256062] IDMAC_CHA_PRI2 =     0x00000000
[   16.259994] IDMAC_BAND_EN1 =     0x00000000
[   16.263915] IDMAC_BAND_EN2 =     0x00000000
[   16.267845] IPU_CHA_DB_MODE_SEL0 =     0x00000001
[   16.272289] IPU_CHA_DB_MODE_SEL1 =     0x00000000
[   16.276732] IPU_CHA_TRB_MODE_SEL0 =     0x00800000
[   16.281271] IPU_CHA_TRB_MODE_SEL1 =     0x00000000
[   16.285801] DMFC_WR_CHAN =     0x00000090
[   16.289557] DMFC_WR_CHAN_DEF =     0x202020F6
[   16.293652] DMFC_DP_CHAN =     0x000096CA
[   16.297400] DMFC_DP_CHAN_DEF =     0x2020F6F6
[   16.301503] DMFC_IC_CTRL =     0x00000002
[   16.305251] IPU_FS_PROC_FLOW1 =     0x00000000
[   16.309442] IPU_FS_PROC_FLOW2 =     0x00000000
[   16.313624] IPU_FS_PROC_FLOW3 =     0x00000000
[   16.317814] IPU_FS_DISP_FLOW1 =     0x00000000
[   16.322000] mxc_ipu mxc_ipu: warning: disable ipu dma channel 0 during its busy state
[   16.331216] In MVC:mxc_free_frames
[   16.888102] In MVC:mxc_v4l_close
[   16.891346] In MVC:mxc_streamoff
[   16.894574] mxc_v4l_close: release resource
[   16.898800] MVC: In mxc_free_frame_buf

[Philip_FSL]

Right, sorry.  I read too quickly.  That does sound strange.  Are you able to capture a snapshot of the signals between the FPGA and the i.MX with an oscilloscope?

[sidebranch]

But I get every second byte duplicated, not every second line.  --- or is this related?

How would I turn of deinterlacing?

[Philip_FSL]

Try turning off de-interlacing.  I think that the de-interlace mode is to duplicate the even fields and discard the odd fields.