iMX6DQ MAX9286 MIPI CSI2 720P camera surround view solution for Linux BSP

[#1]

For iMX6DQ, there are two IPUs, so they can support up to 4 cameras at the same time. But the default BSP can only support up to two cameras at the same time.

The attached patch can make the BSP support up to 4 cameras based on 3.10.53 GA 1.1.0 BSP.

The 4 cameras can be:

- 1xCSI, 3xMIPI

- 2xCSI, 2xMIPI

- 4xMIPI

For 4xMIPI case, the four cameras should be combined on the single MIPI CSI2 interface, and each camera data should be transfered on a mipi virtual channel.

In this patch, we given the example driver for Maxim MAX9286, it was verified working on iMX6DQ SabreAuto board. The input to MAX9286 is four 720P30 cameras.

The verified camera boards:

    (1) Onsemi AR0140+AP0101+MAX9271 boards.

    (2) OmniVision OV10635+MAX9271 boards.

The MIPI CSI2 CVBS camera surround view solution can be found at: iMX6DQ ISL79985/79987 MIPI CSI2 CVBS camera surround view solution for Linux BSP

The MIPI CSI2 CVBS HD camera surround view solution can be found at: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP

The kernel patches:
0001-IPU-update-IPU-capture-driver-to-support-up-to-four-.patch
     Updated IPU common code to support up to four cameras.

0002-Add-Max9286-support-on-SabreAuto-board-which-can-sup.patch
     MAX9286 driver, it includes MAX9271, AP0101 and AR0140 drivers.

0003-Remove-the-page-size-align-requirement-for-v4l2-capt.patch
     With this patch, the mxc_v4l2_tvin test application can use overlay framebuffer as V4l2 capture buffer directly.

0004-Max9286-skip-AP0101-camera-re-initialization.patch
     If the camera board's power had been kept after initialized, this patch will bypass the re-initialization to reduce the start up time.

0005-Max9286-set-I2C-speed-to-400Kbps.patch
    Set I2C to 400Kbps to reduce the AP0101+AR0140 initialization time.

0006-Max9286-add-retry-for-MAX9271-I2C-access.patch
    Added retry for MAX9271 I2C access.

0007-Max9286-Add-support-for-OV10635-camera.patch
    Updated code for OV10635 camera.

0008-Max9286-support-auto-detect-camera-number.patch
    Make the Max9286 driver can detect the camera number automatically.

How to builld the kernel with MAX9286 support:

      make imx_v7_defconfig

      make menuconfig (In this command, you should select the MAX9286 driver:

            Device Drivers  --->

                  <*> Multimedia support  --->

                        [*]   V4L platform devices  --->

                              <*>   MXC Video For Linux Video Capture

                                      MXC Camera/V4L2 PRP Features support  --->

                                          <*>Maxim max9286 GMSL Deserializer Input support

                                              Select Camera Sensor (OmniVision OV10635 camera sensor)  // Or (Onsemi AP0101 and AR0140 camera sensor)

                                          <*>mxc VADC support

                                          <*>Select Overlay Rounting (Queue ipu device for overlay library)

                                          <*>Pre-processor Encoder library

                                          <*>IPU CSI Encoder library)

      make zImage

      make dtbs

The built out image file:

      arch/arm/boot/dts/imx6q-sabreauto.dtb

      arch/arm/boot/zImage

"mxc_v4l2_tvin_max9286.tar.gz" is the test application, test command to capture the four cameras and render on 1080P HDMI display:

/mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d &

/mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d &

/mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d &

/mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &

Some hardware check point on AR0140+AP0101+MAX9271 camera board (Please get MAX9286 and OV10635 schematics from Maxim):

1. In this patch, MAX9286's I2C address is 0x4D, so ADD0 and ADD1 should be connected to high.

AP0101's I2C address is 0xBA, so SADDR should be connected to high.

2. AP0101's DOUT0~DOUT7 should be connected to MAX9271's DIN7~DIN0, the order should be switched, MSB connected to LSB.

3. MAX9271's GPO pin should be connected to AP0101's FRAME_SYNC pin. The pull down resistance on FRAME_SYNC pin should not be 0 ohm.

Some known limitation:

1. AP0101's VSYNC invalid time, last video line's HSYNC to VSYNC porch's max value is 255 pixel clocks, it is not enough for MAX9286 to generate the Frame End MIPI packets for each camera. So in order to let iMX6DQ to capture 1280x720 video for each camera, we had let AP0101 output 1280*724 frame size, and iMX6 will only capture 720 lines, the remained video data and Frame End will be ignored.

This solution will not impact the function, but there will be "Error matching Frame Start with Frame End for Virtual Channel x" error reported from iMX6 MIPI_CSI_ERR1 register.

Maxim suggested to use MAX96705 to relace the MAX9271, it can delay the VSYNC invalid time, then the MIPI error will be fixed.

2015-11-17 update: Updated for OV10635 camera support.

File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-11-17.zip

2015-12-04 update:

File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-12-04.zip

Added patch 0009-Max9286-updated-PCLK-edge-setting-for-OV10635.patch to correct the OV10635 PCLK edge setting

2016-03-07 update:

File L3.14.38_GA_MAX9286_Surroundview_Patch_2016-03-07.zip

Added kernel patch for L3.14.38 GA 1.1.0 BSP.

2016-07-26 update:

Files: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.38_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip.

Added gstreamer support.

Added MAX96705 support.

Added patch for L3.14.52_GA1.1.0.

2017-12-11 update:

Added CVBS surround view link: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP

2021-04-26 update:

Some customer reported, when system loading is heavy, sometimes, some camera will flicker left and right. It is caused by SFMC FIFO data lost. The original patch used IDMAC 0 and IDMAC 1 for two cameras on one IPU, this is not the best setting. 

IDMAC 1 is fixed to use 1/4 SMFC FIFO and it will cause IDMAC 0 to use 1/4 SMFC FIFO too. And another 1/2 of SMFC FIFO can't be used in this case.

Some code update to improve it:

For each IPU, please use IDMAC 0 and IDMAC 2 to capture the two cameras.
This needs change the hard coding in "drivers\media\platform\mxc\capture\ipu_csi_enc.c",
"CSI_MEM1" and "IPU_IRQ_CSI1_OUT_EOF" should be changed to "CSI_MEM2" and "IPU_IRQ_CSI2_OUT_EOF". In this case, all SMFC FIFO can be used.

And in "ipu_common.c", function ipu_probe(), the followed code should be changed to make IDMAC2 use high priority too.
/* Set sync refresh channels and CSI->mem channel as high priority */
- ipu_idmac_write(ipu, 0x18800003L, IDMAC_CHA_PRI(0));
+ ipu_idmac_write(ipu, 0x1880000FL, IDMAC_CHA_PRI(0));

[dqz]

Hi li

i use max9286 and max96705

i found the connect is not very stable, sometime i get "mipi csi2 can not receive sensor clk..." ,when i  git this error i fount all the config are success

i use the patch L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip,

in the patch i found you config max9286 register 0x3f 0x3b ,but i did not found any introduction about these register ?

[qiang_li-mpu_se]

The Max9286 setting is from "MAX9286 Programming Guide", I think you need check your hardware, if the MAX9286 can't lock the camera signals, it will not output valid MIPI CSI2 signal, then on iMX6 side, it will show "mipi csi2 can not receive sensor clk".

[dqz]

would you please send the MAX9286 Programming Guide to me

[dqz]

it like the signle is not stable , 70% it can work succefully ,the  hard ware is designed according to the official schematic design.I am not sure where to check

[qiang_li-mpu_se]

I think you need check with Maxim for both the MAX9286 Programming Guider and the MAX9286 hardware schematics. From my point, the power supply to camera board with Coax cable is easy to cause such stable issue.

[dqz]

i used the patch L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-12-04.zip , when i connect 2 camera , i can capture image correctly. but when i connect three cameras i can not get image correctly, the image scroll.

[qiang_li-mpu_se]

The MAX9286 needs all camera working at frame sync mode and same timing, so the cameras should send out vsync at almost same time. If some camera is working at free run mode, there will be scroll issue, because MAX9286 can't sync them.

You can measue the VSYNC for all cameras to check this. 

[chrisroed]

Hi Qiang,

This is great information!  In the event that you want to use 2 CSI2 cameras, do you know if it's possible to hook both cameras directly to the processor (how do you handle the 2 clk pairs from the two cameras?) of if you need to use a signal aggregator to combine the cameras into one interface?

[qiang_li-mpu_se]

Hi Chris, if they are two MIPI CSI2 cameras, you can't connect them to CPU directly.

[qilinhe]

Hi Qiang,

I use max9286 and max96705+ov10635 on imx8x kernel version is 4.9.51, Does this version have patch?

Use the default max9286 driver which exist in kernel  4.9.51 source code，It show some error when driver load。

It seems ov10635 Communication failure by I2C， But the max9286 is success use the same I2C access

error log as below：

[    2.525043] mxc-mipi-csi2 58227000.csi: mipi_csi2_probe

[    2.549343] mxc-mipi-csi2 58227000.csi: lanes: 4, name: mxc-mipi-csi2.0

[    2.556705] max9286_mipi 0-006a: In max9286_hardware_preinit()

[    2.854197] max9286_mipi: reg = 0xf0.

[    2.857870] max9286_mipi: sensor number = 4.

[    2.862158] max9286_mipi 0-006a: In max9286_hardware_init()

[    2.893273] max9286_mipi: initialized sensor  = 0x01.

[    2.921271] max9286_mipi: initialized sensor  = 0x02.

[    2.949272] max9286_mipi: initialized sensor  = 0x03.

[    2.977271] max9286_mipi: initialized sensor  = 0x04.

[    3.145279] max9286_mipi 0-006a: ov10635_read_reg:read reg error: reg=0x300a.

[    3.152433] max9286_mipi 0-006a: ov10635_check_device: OV10635 hasn't been found, reg[0x300a] = 0x0., index=1

[    3.321275] max9286_mipi 0-006a: ov10635_read_reg:read reg error: reg=0x300a.

[    3.328419] max9286_mipi 0-006a: ov10635_check_device: OV10635 hasn't been found, reg[0x300a] = 0x0., index=2

[    3.497275] max9286_mipi 0-006a: ov10635_read_reg:read reg error: reg=0x300a.

[    3.504421] max9286_mipi 0-006a: ov10635_check_device: OV10635 hasn't been found, reg[0x300a] = 0x0., index=3

[    3.673275] max9286_mipi 0-006a: ov10635_read_reg:read reg error: reg=0x300a.

[    3.680420] max9286_mipi 0-006a: ov10635_check_device: OV10635 hasn't been found, reg[0x300a] = 0x0., index=4

[    4.752679] max9286_mipi 0-006a: max9286_mipi is found, name max9286_mipi 0-006a

Could you help me which part error will lead to ov10635 access failure.

[qiang_li-mpu_se]

The I2C access issue is more like hardware issue, I think you need check your hardware.

The default iMX8 BSP supports MAX9286 directly, no patch is needed.

[qilinhe]

Hi Qiang,

We use the LI-USB30-OV10635-SER  connect to PC by usb 3.0,  the capture show normal on tools, We think the ov10635 modules is OK.

max9286 driver load,  max9286 can access by I2C, I2C driver I think is OK.

We doubt the configure of max9286 and max96705，Is this combination be worked on imx8x 4.9.51 version？

[qiang_li-mpu_se]

The camera we verified on iMX8 is MAX9271+OV10635, but if you are using the MAX96705, you can reference to this iMX6 patch, there is only some small modifications for MAX96705.

Your failure means when MAX9286 try to access OV10635 with Coax cable (I2C signals are transfered on it), there is I2C access error, the I2C between MAX9286 and iMX8 has no error. There is is between MAX9286 and camera module. It is more like Coax signal issue.

[liyu1]

zhang hongliang :

                   你好。我的 sensor跟你是一样的ov10640+ ov490 ISP  + max96705 ，请问一下。您的调试过的代码能发给我一份吗？ 或者 在以前官方提供的代码 上，我还要改哪些参数或者寄存器？

                   谢谢

[bensonhuang]

Hi Qiang Li,

We know that IMX8 BSP has support for MAX9286, but it seems only for i.MX8Q.

MAX9286 driver must work with MIPI CSI2 driver.

In i.MX8Q device tree (fsl-imx8qm-mek.dts), we see it is linked with 'fsl,mxc-mipi-csi2' driver (drivers/media/platform/imx8/mxc-mipi-csi2.c).

But in i.MX8M device tree, MIPI CSI2 driver is 'fsl,mxc-mipi-csi2_yav' (drivers/media/platform/imx8/mxc-mipi-csi2-yav.c).

After comparing these two drivers, we found the major differences are the hardware manipulation and V4L2 media control pad initialization.

Our questions are:

  1. Does 'fsl,mxc-mipi-csi2' driver work on i.MX8M?

  2. Otherwise, does the 'fsl,mxc-mipi-csi2_yav' driver support virtual channel well?

  3. Furthermore, how to support virtual channel feature in 'fsl,mxc-mipi-csi2_yav' driver? (the i.MX8M reference manual does not have enough information for MIPI CSI2 virtual channel)?

Thanks.

[beijianfeng]

hi liqiang,

two question about the test mxc_v4l2_tvin_max9286.tar,the screen resolution is 800x480

when i use the command "mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 800 -oh 480 -d 1 -x 0" ,the camera seem normal

1.when i use the command "mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 400 -oh 240 -d 1 -x 0"  the screen will scroll 

2.when i use the command " mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 800 -oh 480 -d 1 -x 0 -g2d", the sreen show fliker 

thank you very much .looking for you answer.

[qiang_li-mpu_se]

The hardware has pixel align requirement, you'd better let the width and height to align in 16 pixels.

Another thing should be checked: the G2D will set frame buffer to RGB format, but without "-g2d", it will use IPU to render and it needs YUV format fb1.

[qiang_li-mpu_se]

You can reference to the mxc_v4l2_tvin in https://community.nxp.com/docs/DOC-335803

It had fixed some issues.

[beijianfeng]

hi liqiang,

   Thank you for advice.  Maybe it is due to i use the imx6d ,not the imx6Q?? if i use the imx6D ,how can i make it right?

tkank you very much for your help.

[beijianfeng]

hi liqiang,

   i use the imx6q instead of imx6d, but the problem still exits. now i have no idea to solve this problem. Is it an application problem or a kernel problem? please help,thank you ,anyway.

[braveban]

大佬，你不是之前都在3.0.35的版本上调出来了么，交流分享一下呗

[pengtaopengtao]

Hi Qiang Li:

    我在L3.14.52_GA1.1.0版 本上调试情况:

1. Onsemi AR0143+AP0101+MAX96705+Max9286+Imx6D 单路显示正常.

2. Onsemi AR0143+AP0101+MAX96705+Max9286+Imx6D 两路显示效果如图所示:

右边一个摄像头显示为什么是这样的,且有四个相同的画面,同时偏绿.

是什么问题呢?

[qiang_li-mpu_se]

Hi Pengtao，

AP0101需要设置成Frame Sync模式，而不能是默认的Free Run模式；在Free Run模式下，就会看到你的现象，单个camera是OK的，多个camera就不行。

如果你的camera设置是在flash ROM里面的，那么这些设置需要按照driver里面进行修改。

[qiang_li-mpu_se]

Hi Jianfeng,

Can you try the mxc_v4l2_tvin from "https://community.nxp.com/docs/DOC-335803", I haven't seen your issue. And you can also modify the mxc_v4l2_tvin.c source code, it can capture video data to file, then you can check the file on PC with some YUVPlayer to check if the captured data is correct or not.

/*
 char still_file[100] = "./still.yuv";
 int fd_still = 0;
 if ((fd_still = open(still_file, O_RDWR | O_CREAT | O_TRUNC, 0x0666)) < 0)
 {
  printf("Unable to create y frame recording file\n");
 }
*/

... ...

//  if(i>100 && i<120)
//   write(fd_still, capture_buffers[capture_buf.index].start, 720*240*2);

... ...

// close(fd_still);

[pengtaopengtao]

Hi Qiang Li:

1. max9286 这边的FRSYNC/GPI 一定要与imx6Ｄ的CSI0_MCLK相连吗？

2.我们现在硬件没有这样连接，可行吗？

3.我们可采用max9286内同步吗？

4.CSI0_MCLK起什么作用？它与max9286 这边的FRSYNC/GPI 之前是什么关系？

[qiang_li-mpu_se]

FRSYNC/GPI不需要连接到iMX6，CSI0_MCLK也并不使用，所有的工作模式设置都是MAX9286和camera之间的，所以具体MAX9286和Camera（AP0101）之间的连接设置，你需要和Maxim那边确认，对于iMX6这边来讲，只需要把MIPI CSI2的data和clock连过来就可以了。

MAX9286和Camera直接工作模式不对的话，MIPI上过来的数据就是异常的。

[pengtaopengtao]

Hi, Qiang Li：

      谢谢！你的回复，我们现在已经实现了两路显示了，并正常显示．

[lemficb]

Hi qiang,

      I am porting the 360 surround view based on imx8qxp, and the kernel is 4.9.51,and the sensor is ap0101+AR0140,the other is max9286+max96705 . I port the driver from imx6q's driver to imx8 just like all right , because the mxa9286 and four ap0101 have been found successfully in the log. But when I use the imx6's test code 'mxc_v4l2_tvin.out ' ,there is a error just like 'no such files found '. I am not sure imx6's test code could be used in the imx8qxp, is there  any other test code just for imx8 ?? If yes ,could you  provide one demo for me ? Thanks a lot.

Best regards

jun

[qiang_li-mpu_se]

iMX8 has its own test application, you can find it from the yocto imx-test-xxx.tar.gz, test/mxc_v4l2_test/mx8_v4l2_cap_xxx.c

[bensonhuang]

Hi Qiang,

It can't work. The error message is as below.

Do you know which one is the correct executable for i.MX8M?

Thanks.

[qiang_li-mpu_se]

For iMX8M, it is not "mx8_v4l2_cap_xxx.c", it is mx6s_v4l2_cap_drm.c, you can find it from the L4.9.51 GA release for IMX8MQ.

[lemficb]

Hi qiang ,

    I use the 'mx8_v4l2_cap_drm.out' test my board, but the test progress is stopped in the 'start_capturing' function. Finally , I found that it actually stopped in the 'ioctl(fd_v4l, VIDIOC_DQBUF, &buf)' function . And then , I test the four output singal of the max9286, Dout0~Dout3 are all have singal, and the clock I tested  is 295MHz,the input also have singal too. It seems that max9286 and sensors are all work well ,but there is nothing showed on my screen. It confused me for some days. Could you give some advieces to me ? Thanks.  The log is attached below.

root@imx8mqevk:~# ./mx8_v4l2_cap_drm.out -cam 3
init_video_channel, /dev/video0 init 0
init_video_channel, /dev/video1 init 1
Open /dev/dri/card0 success
num of fb:0
num of crtc:2
num of encoder:1
num of connector:1
num of modes = 1
num of encoders = 1
num of props = 2[ 21.059227] mxc-mipi-csi2 58227000.csi: width=1280, height=800, fmt.code=0x2011

modes[0] info:resolution=1024*600 pixels
kms info: fb_base = 0[ 21.071187] input fmt YUV4
x789b3000
w/h=(1024,600)
bits_per_pixel=4
bpp=32
screen_buf_s[ 21.079551] output fmt RGB4
ize =2457600
xres=1024, y_res=600
ch_id=0, w=512, h=300, x_offs[ 21.088947] mxc-isi 58100000.isi: input_size(800,1280), output_size(300,512)
et=0, y_offset=0
ch_id=1, w=512, h=300, x_offset=512, y_offset=0[ 21.105981] mxc-mipi-csi2 58227000.csi: width=1280, height=800, fmt.code=0x2011

Try to open device /dev/video0
cap=0x84201000
index=0
pixelf[ 21.113602] input fmt YUV4
ormat (output by camera): RGBP
index=1
pixelformat (output by c[ 21.121958] output fmt RGB4
amera): RGB3
index=2
pixelformat (output by camera): RGB4
inde[ 21.130400] mxc-isi 58110000.isi: input_size(800,1280), output_size(300,512)
x=3
pixelformat (output by camera): BGR3
index=4
pixelformat (output by camera): BA24
index=5
pixelformat (output by camera)[ 21.150628] mxc-mipi-csi2 58227000.csi: width=1280, height=800, fmt.code=0x2011
: YUYV
index=6
pixelformat (output by camera): YUV4
VIDIOC ENUM FMT failed, index=7
video_ch=0, width=512, height=300,
pixelformat: RGB4
WxH@fps = 512x300@30 Image size = 614400
v4l_capture_setup, Open v4l_dev=0x4, channel=0
Try to open device /dev/video1
cap=0x84201000
index=0
pixelformat (output by camera): RGBP
index=1
pixelformat (output by camera): RGB3
index=2
pixelformat (output by camera): RGB4
index=3
pixelformat (output by camera): BGR3
index=4
pixelformat (output by camera): BA24
index=5
pixelformat (output by camera): YUYV
index=6
pixelformat (output by camera): YUV4
VIDIOC ENUM FMT failed, index=7
video_ch=1, width=512, height=300,
pixelformat: RGB4
WxH@fps = 512x300@30 Image size = 614400
v4l_capture_setup, Open v4l_dev=0x5, channel=1
buffer[0] startAddr=0x7891d000, offset=0x0, buf_size=614400
buffer[1] startAddr=0x78887000, offset=0x96000, buf_size=614400
buffer[2] startAddr=0x787f1000, offset=0x12c000, buf_size=614400
buffer[0] startAddr=0x7875b000, offset=0x0, buf_size=614400
buffer[1] startAddr=0x786c5000, offset=0x96000, buf_size=614400
buffer[2] startAddr=0x7862f000, offset=0x12c000, buf_size=614400
start_capturing channel=0, v4l_dev=0x4
start_capturing channel=1, v4l_dev=0x5

[gokcengokceoglu]

Hi Qiang Lee!

I am using ap0101 with max96705 and max9286 deserializer. I was able to establish connection with max9286 and max96705. However, while reading max96705 registers, I get the following error:

 [ 21.113432] max9286_mipi 1-006a: max9271_read_reg:read reg error: reg=35
[ 21.120143] MAX9271 Reg 0x35 = 0xffffffff.
[ 21.323460] max9286_mipi 1-006a: max9271_read_reg:read reg error: reg= a
[ 21.330183] MAX9271 Reg 0x0a = 0xffffffff.
[ 21.533474] max9286_mipi 1-006a: max9271_read_reg:read reg error: reg= b
[ 21.540199] MAX9271 Reg 0x0b = 0xffffffff.
[ 21.743480] max9286_mipi 1-006a: max9271_read_reg:read reg error: reg= c
[ 21.750201] MAX9271 Reg 0x0c = 0xffffffff.

Any help would be appreciated ! 

[qiang_li-mpu_se]

I think you need check the signal quality on Coax between MAX9286 and MAX96705.

[jiujinhong]

Hi Qiang Li,

1xCSI, 3xMIPI,  what does it meanning?

[jiujinhong]

Hi qiang,

The 4 cameras can be:

- 1xCSI, 3xMIPI

- 2xCSI, 2xMIPI

- 4xMIPI

可以解释一下上面的CSI和MIPI什么意思吗？不知道这个csi是指ipu内部的csi0-3 还是camera mipi总线。

[gokcengokceoglu]

Hi Qiang Li,

We have tried with the following equipment : 

ONSEMI MARS Reference Desşgn (MAX96705, AP0101, AR0140)

30 cm RG174 (and also 3 m coaxial cable) included in the reference design. 

MAX9286S32V Daugter board is used to connect iMX6 board. 

I have taken AP0101 to host configuration mode by removing flash.

Despite I am writing 0x36 to MAX9286' s 34th register to disable local auto acknowledge, I observed that 9th bit coming immediately after 8th bit of SCL. I am attaching the oscilloscope view. The problematic bits are circled with red. 

MAX9286 LFLT led is ON and LOCK led is  OFF.  

Down below, we tried to read from AP0101 (address: 0x92). 

If you have any suggestions it would be appreciated. 

Regards

[gokcengokceoglu]

Hi Qiang Lee,

I have tried initializing MAX9286 in high immunity mode and now, ar140 is being found. However, I think there is something wrong with the v4l part. Here is the related log : 

[ 34.085201]
[ 34.086911] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[ 34.092313] Modules linked in: wlcore mxc_v4l2_capture ipu_bg_overlay_sdc ipu_still ipu_prp_enc ipu_csi_enc max9286_gmsl ipu_fg_overlay_sdc v4l2_int_device wlcore_sdio mxc_dcic
[ 34.108239] CPU: 3 PID: 402 Comm: v4l_id Not tainted 4.1.15 #1
[ 34.114078] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 34.120628] task: d87b2ac0 ti: d8c78000 task.ti: d8c78000
[ 34.123891] max9286_mipi 1-006a: max9286_write_reg:write reg error:reg=3b,val=1e
[ 34.133488] PC is at mxc_v4l_do_ioctl+0xc4c/0x2590 [mxc_v4l2_capture]
[ 34.139979] LR is at mxc_v4l_do_ioctl+0xc3c/0x2590 [mxc_v4l2_capture]
[ 34.146476] pc : [<7f08539c>] lr : [<7f08538c>] psr: 600e0013
[ 34.146476] sp : d8c79e40 ip : 00000074 fp : 00000000
[ 34.157991] r10: d88f9300 r9 : 7f034094 r8 : d8c79e90
[ 34.163245] r7 : 80685600 r6 : 00000000 r5 : d8d19008 r4 : d8d19000
[ 34.169792] r3 : 00000000 r2 : d8c79ea7 r1 : 7f087ae0 r0 : d8c79ec0
[ 34.176346] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 34.183504] Control: 10c53c7d Table: 6939c04a DAC: 00000015
[ 34.189288] Process v4l_id (pid: 402, stack limit = 0xd8c78210)
[ 34.195251] Stack: (0xd8c79e40 to 0xd8c7a000)
[ 34.199652] 9e40: d88f9300 00000000 00000000 00000000 d8868550 d6023000 d80fe028 d8c41700
[ 34.207872] 9e60: 80b154c0 80685600 00000068 00000000 00000002 7ebeec88 d8c79e90 7f084750
[ 34.216090] 9e80: 00000000 804dbc10 d88f9300 800f6978 5f63786d 326c3476 00000000 00000000
[ 34.224327] 9ea0: 3978616d 00363832 00000000 00000000 00000000 00000000 00000000 00000000
[ 34.232540] 9ec0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 34.240757] 9ee0: 0004010f 85200005 05200005 00000000 00000000 00000000 9087e695 00000006
[ 34.248978] 9f00: d925f015 76f11000 00000000 d61505d8 d8ffa318 7ebeec88 d8ffa318 d88f9300
[ 34.253976] max9286_mipi: sensor number = 1.
[ 34.254788] mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: mipi_lane_bps = 148 Mbps
[ 34.254801] mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: value = 0x42.
[ 34.276213] 9f20: 00000003 7ebeec88 d8c78000 00000000 00000000 800fa0e4 d8c79f40 800e3970
[ 34.284441] 9f40: 80b06720 00000003 d88f9300 d925f000 d8ffa318 d88f9308 00000020 00000000
[ 34.292668] 9f60: 00000000 800e91ac 00000000 00000000 d88f9300 00000003 d88f9300 80685600
[ 34.300888] 9f80: 7ebeec88 d8c78000 00000000 800fa338 00000000 7ebeee54 00000003 00000036
[ 34.309102] 9fa0: 8000f604 8000f480 00000000 7ebeee54 00000003 80685600 7ebeec88 00000001
[ 34.317312] 9fc0: 00000000 7ebeee54 00000003 00000036 00000000 00000000 76f11000 00000000
[ 34.325534] 9fe0: 76e6e2a0 7ebeec7c 00011cfc 76e6e2ac 600e0010 00000003 8112f040 0420f044
[ 34.333788] [<7f08539c>] (mxc_v4l_do_ioctl [mxc_v4l2_capture]) from [<804dbc10>] (video_usercopy+0x1b8/0x480)
[ 34.343777] [<804dbc10>] (video_usercopy) from [<800fa0e4>] (do_vfs_ioctl+0x3e8/0x608)
[ 34.351752] [<800fa0e4>] (do_vfs_ioctl) from [<800fa338>] (SyS_ioctl+0x34/0x5c)
[ 34.359115] [<800fa338>] (SyS_ioctl) from [<8000f480>] (ret_fast_syscall+0x0/0x3c)
[ 34.366711] Code: e5993008 e3071ae0 e2880030 e3471f08 (e5932018)
[ 34.374557] ---[ end trace 756e286723ff907f ]---
[ 34.435645] FAT-fs (mmcblk1p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[ 34.531108] kjournald starting. Commit interval 5 seconds
[ 34.549007] EXT4-fs (mmcblk1p2): recovery complete
[ 34.553859] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
[ 34.591916] EXT3-fs (mmcblk0p2): using internal journal
[ 34.597183] EXT3-fs (mmcblk0p2): recovery complete
[ 34.601980] EXT3-fs (mmcblk0p2): mounted filesystem with ordered data mode
[ 34.602407] FAT-fs (mmcblk0p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[ 34.644868] wlcore: loaded
[ 34.733493] max9286_mipi 1-006a: ap0101_initialize: AP0101 was found, index = 1.
[ 35.357742] ap0101_LastPatchStatus
[ 35.812239] ap0101_LastPatchStatus
[ 35.985973] ap0101_LastPatchStatus
[ 36.194854] ap0101_LastPatchStatus
[ 36.243936] ap0101_LastPatchStatus
[ 36.392576] ap0101_LastPatchStatus
[ 37.086270] max9286_mipi 1-006a: ap0101_InitSensor: AR0140 was found, index = 1.
[ 94.823493] udevd[215]: worker [230] /devices/soc0/v4l2_cap_1/video4linux/video1 is taking a long time
[ 95.098316] udevd[230]: slow: 'v4l_id /dev/video1' [402]
bootlogd: cannot allocate pseudo tty: No such file or directory

[qiang_li-mpu_se]

As I remembered, the blank from frame end to new frame is not enough for AP0101, so even the camera output resolution is 1280*800, we need capture just 1280*720, otherwise, the video capture will fail.

[gokcengokceoglu]

Hi!

Thank you for the reply. The resolution is 1280x720. So, I think that is not the problem. We are using kernel version 4.1.15. 

May the following errors be related to that?

[ 36.921793] v4l2_int_device wlcore_sdio mxc_dcic
[ 36.926813] CPU: 3 PID: 399 Comm: v4l_id Not tainted 4.1.15 #1

[ 94.823493] udevd[215]: worker [230] /devices/soc0/v4l2_cap_1/video4linux/video1 is taking a long time
[ 95.098316] udevd[230]: slow: 'v4l_id /dev/video1' [402]
bootlogd: cannot allocate pseudo tty: No such file or directory

Have a nice day!

[gokcengokceoglu]

Hi Qiang Li !

Is it possible for you to share "60-persistent-v4l.rules" file? It should be inside /lib/udev/rules.d

Thank you very much

Regards,

Gokcen 

[pengtaopengtao]

Hi Qiang Li !

   imx8 可以支持几路720P 60fps的摄像头输入呢？（所有IMX8在内）

[qiang_li-mpu_se]

标准的720P60的pixel clock是74.25MHz，如果是UYVY 16bpp的camera，那么单个camera的带宽就是74.25M*16 = 1.188Gbps

iMX8，iMX8X的MIPI CSI2最大带宽都是6Gbps，每个可以支持4个virtual channel，所以每个MIPI CSI2口可以支持4路720P60的camera，iMX8有2个MIPI CSI2口，所以可以支持8个。iMX8X只有一个MIPI CSI2，所以可以支持4路，但是它有一个8位的parallel CSI，也可以支持一路720P camera，加起来就是5路。

iMX8M虽然有2个MIPI CSI2，但是它不支持multi virtual channel，所以只能支持2路camera。

[wenxiaofeng]

Hi. Qiang Li

The four camera`s video can render on 1080P HDMI display. 

How does the surround view algorithm get the four cameras`s capture buffers.

do you have some good ideas?

[pengtaopengtao]

Hi. Qiang Li

      当我们接入4路 720p@60pfs的camera时，从摄像头接入到在屏上显示时的延迟是多少？这个性能指标你们有没有呢？

      当我们接入8路 720p@60pfs的camera时，从摄像头接入到在屏上显示时的延迟是多少？这个性能指标你们有没有呢？

[2479420618]

Hi Li Qiang

      Thank you for your Patch.

       Now,I have some trouble when I use "patch".

       Can you write a manual to me so that I can understand how to use the "patch".

       Thanks Sincerely.

       Zhongmao Liu.

[qiang_li-mpu_se]

From real test, the total delay is about 86ms.

Camera number will not impact the delay, capture delay is fixed on camera fps, for example, 30fps camera input, the capture delay is 33ms. And render delay is fixed on display fps, for examlpe, 60Hz display, the delay is 17ms, then the post-process delay, it needs less than camera fps (33ms) to avoid frame lost. So for 8 camera case, you need make sure all the post process is less than 33ms to avoid frame drop.

[2479420618]

HI Qiang Li

  When I use the "Patch",my kernel show error.My kernel vision is kernel-3.10.53.

  So,I want to now what's the differance between the kernel-3.10.53 and L3.10.53_1.1.0-GA.

  I think that the different vision make the error.

[2479420618]

Hi Qiang Li,

  The test application file not contain "g2d.h",but  "mxc_v4l2_tvin.c" has the code #include "g2d.h".So,when I make it,it turn out error.

  Can you help me solve the problem?

  Think you!

[qiang_li-mpu_se]

The head file is in your rootfs which is built from Yocto.

You can also use the binary "mxc_v4l2_tvin.out" directly.