For iMX6DQ, there are two IPUs, so they can support up to 4 cameras at the same time. But the default BSP can only support up to two cameras at the same time.
The attached patch can make the BSP support up to 4 cameras based on 3.10.53 GA 1.1.0 BSP.
The 4 cameras can be:
- 1xCSI, 3xMIPI
- 2xCSI, 2xMIPI
- 4xMIPI
For 4xMIPI case, the four cameras should be combined on the single MIPI CSI2 interface, and each camera data should be transfered on a mipi virtual channel.
In this patch, we given the example driver for Maxim MAX9286, it was verified working on iMX6DQ SabreAuto board. The input to MAX9286 is four 720P30 cameras.
The verified camera boards:
(1) Onsemi AR0140+AP0101+MAX9271 boards.
(2) OmniVision OV10635+MAX9271 boards.
The MIPI CSI2 CVBS camera surround view solution can be found at: iMX6DQ ISL79985/79987 MIPI CSI2 CVBS camera surround view solution for Linux BSP
The MIPI CSI2 CVBS HD camera surround view solution can be found at: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP
The kernel patches:
0001-IPU-update-IPU-capture-driver-to-support-up-to-four-.patch
Updated IPU common code to support up to four cameras.
0002-Add-Max9286-support-on-SabreAuto-board-which-can-sup.patch
MAX9286 driver, it includes MAX9271, AP0101 and AR0140 drivers.
0003-Remove-the-page-size-align-requirement-for-v4l2-capt.patch
With this patch, the mxc_v4l2_tvin test application can use overlay framebuffer as V4l2 capture buffer directly.
0004-Max9286-skip-AP0101-camera-re-initialization.patch
If the camera board's power had been kept after initialized, this patch will bypass the re-initialization to reduce the start up time.
0005-Max9286-set-I2C-speed-to-400Kbps.patch
Set I2C to 400Kbps to reduce the AP0101+AR0140 initialization time.
0006-Max9286-add-retry-for-MAX9271-I2C-access.patch
Added retry for MAX9271 I2C access.
0007-Max9286-Add-support-for-OV10635-camera.patch
Updated code for OV10635 camera.
0008-Max9286-support-auto-detect-camera-number.patch
Make the Max9286 driver can detect the camera number automatically.
How to builld the kernel with MAX9286 support:
make imx_v7_defconfig
make menuconfig (In this command, you should select the MAX9286 driver:
Device Drivers --->
<*> Multimedia support --->
[*] V4L platform devices --->
<*> MXC Video For Linux Video Capture
MXC Camera/V4L2 PRP Features support --->
<*>Maxim max9286 GMSL Deserializer Input support
Select Camera Sensor (OmniVision OV10635 camera sensor) // Or (Onsemi AP0101 and AR0140 camera sensor)
<*>mxc VADC support
<*>Select Overlay Rounting (Queue ipu device for overlay library)
<*>Pre-processor Encoder library
<*>IPU CSI Encoder library)
make zImage
make dtbs
The built out image file:
arch/arm/boot/dts/imx6q-sabreauto.dtb
arch/arm/boot/zImage
"mxc_v4l2_tvin_max9286.tar.gz" is the test application, test command to capture the four cameras and render on 1080P HDMI display:
/mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d &
/mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d &
/mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d &
/mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &
Some hardware check point on AR0140+AP0101+MAX9271 camera board (Please get MAX9286 and OV10635 schematics from Maxim):
1. In this patch, MAX9286's I2C address is 0x4D, so ADD0 and ADD1 should be connected to high.
AP0101's I2C address is 0xBA, so SADDR should be connected to high.
2. AP0101's DOUT0~DOUT7 should be connected to MAX9271's DIN7~DIN0, the order should be switched, MSB connected to LSB.
3. MAX9271's GPO pin should be connected to AP0101's FRAME_SYNC pin. The pull down resistance on FRAME_SYNC pin should not be 0 ohm.
Some known limitation:
1. AP0101's VSYNC invalid time, last video line's HSYNC to VSYNC porch's max value is 255 pixel clocks, it is not enough for MAX9286 to generate the Frame End MIPI packets for each camera. So in order to let iMX6DQ to capture 1280x720 video for each camera, we had let AP0101 output 1280*724 frame size, and iMX6 will only capture 720 lines, the remained video data and Frame End will be ignored.
This solution will not impact the function, but there will be "Error matching Frame Start with Frame End for Virtual Channel x" error reported from iMX6 MIPI_CSI_ERR1 register.
Maxim suggested to use MAX96705 to relace the MAX9271, it can delay the VSYNC invalid time, then the MIPI error will be fixed.
2015-11-17 update: Updated for OV10635 camera support.
File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-11-17.zip
2015-12-04 update:
File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-12-04.zip
Added patch 0009-Max9286-updated-PCLK-edge-setting-for-OV10635.patch to correct the OV10635 PCLK edge setting
2016-03-07 update:
File L3.14.38_GA_MAX9286_Surroundview_Patch_2016-03-07.zip
Added kernel patch for L3.14.38 GA 1.1.0 BSP.
2016-07-26 update:
Files: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.38_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip.
Added gstreamer support.
Added MAX96705 support.
Added patch for L3.14.52_GA1.1.0.
2017-12-11 update:
Added CVBS surround view link: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP
2021-04-26 update:
Some customer reported, when system loading is heavy, sometimes, some camera will flicker left and right. It is caused by SFMC FIFO data lost. The original patch used IDMAC 0 and IDMAC 1 for two cameras on one IPU, this is not the best setting.
IDMAC 1 is fixed to use 1/4 SMFC FIFO and it will cause IDMAC 0 to use 1/4 SMFC FIFO too. And another 1/2 of SMFC FIFO can't be used in this case.
Some code update to improve it:
For each IPU, please use IDMAC 0 and IDMAC 2 to capture the two cameras.
This needs change the hard coding in "drivers\media\platform\mxc\capture\ipu_csi_enc.c",
"CSI_MEM1" and "IPU_IRQ_CSI1_OUT_EOF" should be changed to "CSI_MEM2" and "IPU_IRQ_CSI2_OUT_EOF". In this case, all SMFC FIFO can be used.
And in "ipu_common.c", function ipu_probe(), the followed code should be changed to make IDMAC2 use high priority too.
/* Set sync refresh channels and CSI->mem channel as high priority */
- ipu_idmac_write(ipu, 0x18800003L, IDMAC_CHA_PRI(0));
+ ipu_idmac_write(ipu, 0x1880000FL, IDMAC_CHA_PRI(0));
这个脚电平是低，但是读了下0x27寄存器的bit7却是1 ：
这个是不是也不正常。
Hi Qiang Li,
I am using Imx6qp. With following commands, the system is working properly.
WIDTH=640
HEIGHT=240
./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow ${WIDTH} -oh ${HEIGHT} -d 0 -x 0 -g2d &
./mxc_v4l2_tvin.out -ol ${WIDTH} -ot 0 -ow ${WIDTH} -oh ${HEIGHT} -d 0 -x 1 -g2d &
./mxc_v4l2_tvin.out -ol 0 -ot ${HEIGHT} -ow ${WIDTH} -oh ${HEIGHT} -d 0 -x 2 -g2d &
./mxc_v4l2_tvin.out -ol ${WIDTH} -ot ${HEIGHT} -ow ${WIDTH} -oh ${HEIGHT} -d 0 -x 3 -g2d &
When I try to use gstreamer with the following pipelines, third and forth cameras are jittering and scrolling.
WINDOW1="window-x-coord=0 window-y-coord=0 window-width=${WIDTH} window-height=${HEIGHT}"
WINDOW2="window-x-coord=${WIDTH} window-y-coord=0 window-width=${WIDTH} window-height=${HEIGHT}"
WINDOW3="window-x-coord=0 window-y-coord=${HEIGHT} window-width=${WIDTH} window-height=${HEIGHT}"
WINDOW4="window-x-coord=${WIDTH} window-y-coord=${HEIGHT} window-width=${WIDTH} window-height=${HEIGHT}"
gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false $WINDOW1 &
gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false $WINDOW2 &
gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video2 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false $WINDOW3 &
gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false $WINDOW4 &
v4l applications are working properly but gstreamer causes the problem seen above. Do you have any idea about this problem?
hi liqiang：
我现在在android6系统上面进行多路camera同时录像，用mMediaRecorder进行录制；
录制一路的时候没有问题，但是录制两路的时候gpu报错：
这个是什么原因？
Hi Qiang Li,
The following two pipelines work normally when run seperately.
- gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video0 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false
- gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video1 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false
However, each pipelines given below (especially with video3) cause sawtooth issue given as attached gif, even if they run seperately.
- gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video2 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false
- gst-launch-1.0 -v -e imxv4l2videosrc device=/dev/video3 ! imxg2dvideosink framebuffer=/dev/fb0 force-aspect-ratio=false
While twin app does not create any problem for each camera.
While using gstreamer, what can cause this situation? Is it about gstreamer patch or IPU2?
