i.MX6 SoloX Parallel CSI Problem

[lukasgasser]

I am working on integrating an Onsemi AR0135CS monochrome camera with the i.MX6 SoloX 17x17 without PCIe (MCIMX6X3EVO10AB, 2N19K Mask) on a custom board. The goal is to capture grayscale images from the sensor, make some calculations and transfer results over BLE.

Hardware configuration

• AR0135 12bit/pixel grayscale connected to parallel CSI2 and I2C2
• No LCD output (deactivated in device tree)
• PF0200 PMIC replaced by a Microchip MIC23450 
• See attachment imx6sx-xxxxx.dts

For 12bpp grayscale support I added the pixel format Y12 to mx6s_capture.c:

// ... 
 }, {
  .name  = "RAWRGB8 (SBGGR8)",
  .fourcc  = V4L2_PIX_FMT_SBGGR8,
  .pixelformat = V4L2_PIX_FMT_SBGGR8,
  .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
  .bpp  = 1,
+ }, {
+  .name           = "Grey12 (Y12 )",
+  .fourcc         = V4L2_PIX_FMT_Y12,
+  .pixelformat    = V4L2_PIX_FMT_Y12,
+  .mbus_code      = MEDIA_BUS_FMT_Y12_1X12,
+  .bpp            = 2,
 }
};‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

// ...

 case V4L2_PIX_FMT_YUYV:
  if (csi_dev->csi_mipi_mode == true)
   width = pix->width;
  else
   /* For parallel 8-bit sensor input */
   width = pix->width * 2;
  break;
+ case V4L2_PIX_FMT_Y12:
+  pr_info("width = %d\n", pix->width);
+  width = pix->width;
+  break;
 default:
  pr_debug("   case not supported\n");
  return -EINVAL;
 }

// ...‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Until now I managed to get the I2C part of the driver working and the parallel signals such as VSYNC, HSYNC and DATA are visible on the oscilloscope. But a soon as the CSI driver tries to read data from the CSI module, it fails.

Trying to read from the sensor using a small test application (based on the v4l2grab.c example, source see attachment)  fails. Using gstreamer gst-launch with imxv4l2src triggers the same fault. The test application generates the following application output:

Debugging starts

Listening on port 10000
Remote debugging from host 192.168.137.1
Process /home/root/grabtest created; pid = 5453
libv4l2: error turning on stream: Timer expired
error 62, Timer expired\n


Child exited with status 1
Application finished with exit code 0.‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

and dmesg output (see the full output attached):

root@xxxxx:~# dmesg
...
imx-sdma 20ec000.sdma: loaded firmware 3.4
ar0135_camera: starting probe
ar0135 1-0010: Found ar0135 chip
ar0135_camera: detected chip
ar0135_camera: finished probe
mx6s-csi 221c000.csi: initialising
CSI: Registered sensor subdevice: ar0135 1-0010
ar0135_camera: registered
FAT-fs (mmcblk2p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
FAT-fs (mmcblk3p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
systemd[1]: dev-disk-by\x2duuid-A43B\x2dBCC5.device: Changed dead -> plugged
systemd[1]: dev-disk-by\x2dpath-platform\x2d2198000.usdhc\x2dpart1.device: Changed dead -> plugged
systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/dev_2dmmcblk2p1_2edevice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=88 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
systemd[1]: dev-ram0.device: Changed dead -> plugged
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
systemd[1]: sys-devices-virtual-block-ram0.device: Changed dead -> plugged
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
IPv6: ADDRCONF(NETDEV_UP): usb0: link is not ready
systemd[1]: dev-ram4.device: Changed dead -> plugged
systemd[1]: sys-devices-virtual-block-ram4.device: Changed dead -> plugged
mx6s-csi 221c000.csi: csi v4l2 busfreq high request.
configfs-gadget gadget: high-speed config #1: c
IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=121 reply_cookie=0 signature=so error-name=n/a error-message=n/a
mx6s-csi 221c000.csi: csi v4l2 busfreq high release.
systemd[1]: dev-ram5.device: Changed dead -> plugged
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
mx6s-csi 221c000.csi: csi v4l2 busfreq high request.
ar0135_camera: g_parm
ar0135_camera: enum_mbus_code
ar0135_camera: enum_mbus_code
mx6s-csi 221c000.csi: No more fmt
ar0135_camera: set_fmt
ar0135_camera: set_fmt
mx6s-csi 221c000.csi: set to pixelformat 'Grey12'
width = 1280
ar0135_camera: g_parm
mx6s-csi 221c000.csi: count=2, size=0
size=2457600
vma start=0x76a4b000, size=2457600, ret=0
vma start=0x767f3000, size=2457600, ret=0
mx6s-csi 221c000.csi: mx6s_videobuf_prepare (vb=0x88ca4200) 0x94300000 0
mx6s-csi 221c000.csi: mx6s_videobuf_prepare (vb=0x88ca4a00) 0x94600000 0
mx6s-csi 221c000.csi: mx6s_videobuf_queue (vb=0x88ca4200) 0x94300000 2457600
mx6s-csi 221c000.csi: mx6s_videobuf_queue (vb=0x88ca4a00) 0x94600000 2457600
timeout when wait for SOF
------------[ cut here ]------------
WARNING: CPU: 0 PID: 696 at /usr/src/kernel/drivers/media/v4l2-core/videobuf2-core.c:1347 vb2_start_streaming+0xd4/0x150
Modules linked in: mx6s_capture mxc_dcic ar0135_camera galcore(O) brcmfmac(O) cfg80211(O) brcmutil(O) compat(O)
CPU: 0 PID: 696 Comm: grabtest Tainted: G           O    4.14.98-imx_4.14.98_2.0.0_ga+g5d6cbea #1
Hardware name: Freescale i.MX6 SoloX (Device Tree)
[<8010ef44>] (unwind_backtrace) from [<8010b49c>] (show_stack+0x10/0x14)
[<8010b49c>] (show_stack) from [<80924ba4>] (dump_stack+0x78/0x8c)
[<80924ba4>] (dump_stack) from [<8012dd94>] (__warn+0xe8/0x100)
[<8012dd94>] (__warn) from [<8012de5c>] (warn_slowpath_null+0x20/0x28)
[<8012de5c>] (warn_slowpath_null) from [<80694ce4>] (vb2_start_streaming+0xd4/0x150)
[<80694ce4>] (vb2_start_streaming) from [<80696a74>] (vb2_core_streamon+0x128/0x170)
[<80696a74>] (vb2_core_streamon) from [<7f14762c>] (mx6s_vidioc_streamon+0x3c/0x98 [mx6s_capture])
[<7f14762c>] (mx6s_vidioc_streamon [mx6s_capture]) from [<80681e1c>] (__video_do_ioctl+0x2e0/0x2e8)
[<80681e1c>] (__video_do_ioctl) from [<80681688>] (video_usercopy+0x70/0x518)
[<80681688>] (video_usercopy) from [<8067df28>] (v4l2_ioctl+0x78/0x154)
[<8067df28>] (v4l2_ioctl) from [<80219234>] (do_vfs_ioctl+0x9c/0x8c4)
[<80219234>] (do_vfs_ioctl) from [<80219a90>] (SyS_ioctl+0x34/0x58)
[<80219a90>] (SyS_ioctl) from [<80107980>] (ret_fast_syscall+0x0/0x54)
---[ end trace 31ad7eeed4523d2f ]---
mx6s-csi 221c000.csi: csi v4l2 busfreq high release.
systemd-journald[114]: Successfully sent stream file descriptor to service manager.
root@xxxxx:~#
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I activated kernel debug output using debug on the kernel command line. Furthermore I compiled mx6s_capture.c with #define DEBUG.

Following the path to the source code, obviously the read from the CSISR register fails (it is always zero).

static int mx6s_csi_enable(struct mx6s_csi_dev *csi_dev)
{
 struct v4l2_pix_format *pix = &csi_dev->pix;

 ...

 local_irq_save(flags);
 for (timeout = 10000000; timeout > 0; timeout--) {
  if (csi_read(csi_dev, CSI_CSISR) & BIT_SOF_INT) { // ALWAYS READS 0x0!!
   val = csi_read(csi_dev, CSI_CSICR3);
   csi_write(csi_dev, val | BIT_DMA_REFLASH_RFF,
     CSI_CSICR3);
   /* Wait DMA reflash done */
   for (timeout2 = 1000000; timeout2 > 0; timeout2--) {
    if (csi_read(csi_dev, CSI_CSICR3) &
     BIT_DMA_REFLASH_RFF)
     cpu_relax();
    else
     break;
   }
   if (timeout2 <= 0) {
    pr_err("timeout when wait for reflash done.\n");
    local_irq_restore(flags);
    return -ETIME;
   }
   /* For imx6sl csi, DMA FIFO will auto start when sensor ready to work,
    * so DMA should enable right after FIFO reset, otherwise dma will lost data
    * and image will split.
    */
   csi_dmareq_rff_enable(csi_dev);
   csi_enable_int(csi_dev, 1);
   csi_enable(csi_dev, 1);
   break;
  } else
   cpu_relax();
 }
 if (timeout <= 0) {
  pr_err("timeout when wait for SOF\n");
  local_irq_restore(flags);
  return -ETIME;
 }
 local_irq_restore(flags);

 return 0;
}
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Reads on the CSI peripheral registers always return zero. I tried to read and display register values bot from the kernel module or from a user application using  the mmap() method boundary devices uses in their devregs-tool. Registers on other peripherals can easily be read out with correct values. But all peripherals on the soc:aips3:spba-bus (csi, pxp, dcic, lcdif, ...) return zero values. IMHO the CSICR1 register on physical address 0x0221_C000 for example should return 4000_0800h on reset.

Interestingly in u-boot the values can be read out properly:

U-Boot 2018.03-imx_v2018.03_4.14.98_2.0.0_ga+g87a19df (May 20 2019 - 15:01:33 +0000)

CPU:   Freescale i.MX6SX rev1.2 996 MHz (running at 792 MHz)
CPU:   Extended Commercial temperature grade (-20C to 105C) at 43C
Reset cause: POR
Model: xxxxx (NXP i.MX6SX)
Board: xxxxx
DRAM:  512 MiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1, FSL_SDHC: 2
Loading Environment from MMC... OK
In:    serial
Out:   serial
Err:   serial
switch to partitions #0, OK
mmc1 is current device
flash target is MMC:1
Net:   Board Net Initialization Failed
No ethernet found.
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot:  0
xxxxx $ md 0x0221c000
0221c000: 40000800 00000000 00000000 00000000    ...@............
0221c010: 00000000 00009600 80004000 00000000    .........@......
0221c020: 00000000 00000000 00000000 00000000    ................
0221c030: 00000000 00000000 00000270 00000000    ........p.......
0221c040: 00000000 00000000 0002d000 00000000    ................
0221c050: 00000000 00000000 00000000 00000000    ................
0221c060: 00000000 00000000 00000000 00000000    ................
0221c070: 00000000 00000000 00000000 00000000    ................
0221c080: 00000000 00000000 00000000 00000000    ................
0221c090: 00000000 00000000 00000000 00000000    ................
0221c0a0: 00000000 00000000 00000000 00000000    ................
0221c0b0: 00000000 00000000 00000000 00000000    ................
0221c0c0: 00000000 00000000 00000000 00000000    ................
0221c0d0: 00000000 00000000 00000000 00000000    ................
0221c0e0: 00000000 00000000 00000000 00000000    ................
0221c0f0: 00000000 00000000 00000000 00000000    ................
xxxxx $
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Now I really have no clue what is to do for getting it to work. Any help will be very appreciated.

Thanks in advance!

** UPDATE 2019/06/21 **

I managed to read the registers in the Kernel. The issue was that no LCDIF was enabled and therefore maybe the whole power-domain was deactivated?

Now the driver reads the registers correctly and I am figuring out why there is still a timeout when waiting for SOF.

Cheers!

[lukasgasser · Accepted solution]

I have finally got it to work!

There were several issues:

1) In order to use the CSI peripherals, at least one LCDIF instance must be enabled in the device tree. Our board does not have any LCD so I left both interfaces disabled.

2) The missing SOF was due to the stream not starting which was a mistake in the AR0135 driver.

3) Had to patch the mx6s_capture.c driver to enable SWAP16_EN and PACK_DIR bits to get correct 16 bit pixels with 12 bit data

4) Had to add 'fsl,two-8bit-sensor-mode;' to the device tree in order to get the full 12 bit depth

Hope this helps someone else.

Cheers!

[variable_andrew]

lukasgasser‌, did you use the PXP in this case to do any conversions? 

If so - how did you convert from your Y12 into a supported format used by the rest of the PXP?

[lukasgasser]

Hi Andrew

Unfortunately I didn't use the PXP because we use the raw 12 bit grayscale image data in our application.

BTW: The patch of mx6s_capture.c is only necessary when storing the image data directly in PGM format (which is obviously big endian instead of little endian what mx6s_capture usually outputs). 

A solution to your question could be not to use Y12 but Y16 format, maybe this format has better support in the rest of the imx v4l2 system.

Hope this helps.

Cheers

Lukas