iMX6DQ MAX9286 MIPI CSI2 720P camera surround view solution for Linux BSP

[#1]

For iMX6DQ, there are two IPUs, so they can support up to 4 cameras at the same time. But the default BSP can only support up to two cameras at the same time.

The attached patch can make the BSP support up to 4 cameras based on 3.10.53 GA 1.1.0 BSP.

The 4 cameras can be:

- 1xCSI, 3xMIPI

- 2xCSI, 2xMIPI

- 4xMIPI

For 4xMIPI case, the four cameras should be combined on the single MIPI CSI2 interface, and each camera data should be transfered on a mipi virtual channel.

In this patch, we given the example driver for Maxim MAX9286, it was verified working on iMX6DQ SabreAuto board. The input to MAX9286 is four 720P30 cameras.

The verified camera boards:

    (1) Onsemi AR0140+AP0101+MAX9271 boards.

    (2) OmniVision OV10635+MAX9271 boards.

The MIPI CSI2 CVBS camera surround view solution can be found at: iMX6DQ ISL79985/79987 MIPI CSI2 CVBS camera surround view solution for Linux BSP

The MIPI CSI2 CVBS HD camera surround view solution can be found at: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP

The kernel patches:
0001-IPU-update-IPU-capture-driver-to-support-up-to-four-.patch
     Updated IPU common code to support up to four cameras.

0002-Add-Max9286-support-on-SabreAuto-board-which-can-sup.patch
     MAX9286 driver, it includes MAX9271, AP0101 and AR0140 drivers.

0003-Remove-the-page-size-align-requirement-for-v4l2-capt.patch
     With this patch, the mxc_v4l2_tvin test application can use overlay framebuffer as V4l2 capture buffer directly.

0004-Max9286-skip-AP0101-camera-re-initialization.patch
     If the camera board's power had been kept after initialized, this patch will bypass the re-initialization to reduce the start up time.

0005-Max9286-set-I2C-speed-to-400Kbps.patch
    Set I2C to 400Kbps to reduce the AP0101+AR0140 initialization time.

0006-Max9286-add-retry-for-MAX9271-I2C-access.patch
    Added retry for MAX9271 I2C access.

0007-Max9286-Add-support-for-OV10635-camera.patch
    Updated code for OV10635 camera.

0008-Max9286-support-auto-detect-camera-number.patch
    Make the Max9286 driver can detect the camera number automatically.

How to builld the kernel with MAX9286 support:

      make imx_v7_defconfig

      make menuconfig (In this command, you should select the MAX9286 driver:

            Device Drivers  --->

                  <*> Multimedia support  --->

                        [*]   V4L platform devices  --->

                              <*>   MXC Video For Linux Video Capture

                                      MXC Camera/V4L2 PRP Features support  --->

                                          <*>Maxim max9286 GMSL Deserializer Input support

                                              Select Camera Sensor (OmniVision OV10635 camera sensor)  // Or (Onsemi AP0101 and AR0140 camera sensor)

                                          <*>mxc VADC support

                                          <*>Select Overlay Rounting (Queue ipu device for overlay library)

                                          <*>Pre-processor Encoder library

                                          <*>IPU CSI Encoder library)

      make zImage

      make dtbs

The built out image file:

      arch/arm/boot/dts/imx6q-sabreauto.dtb

      arch/arm/boot/zImage

"mxc_v4l2_tvin_max9286.tar.gz" is the test application, test command to capture the four cameras and render on 1080P HDMI display:

/mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d &

/mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d &

/mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d &

/mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &

Some hardware check point on AR0140+AP0101+MAX9271 camera board (Please get MAX9286 and OV10635 schematics from Maxim):

1. In this patch, MAX9286's I2C address is 0x4D, so ADD0 and ADD1 should be connected to high.

AP0101's I2C address is 0xBA, so SADDR should be connected to high.

2. AP0101's DOUT0~DOUT7 should be connected to MAX9271's DIN7~DIN0, the order should be switched, MSB connected to LSB.

3. MAX9271's GPO pin should be connected to AP0101's FRAME_SYNC pin. The pull down resistance on FRAME_SYNC pin should not be 0 ohm.

Some known limitation:

1. AP0101's VSYNC invalid time, last video line's HSYNC to VSYNC porch's max value is 255 pixel clocks, it is not enough for MAX9286 to generate the Frame End MIPI packets for each camera. So in order to let iMX6DQ to capture 1280x720 video for each camera, we had let AP0101 output 1280*724 frame size, and iMX6 will only capture 720 lines, the remained video data and Frame End will be ignored.

This solution will not impact the function, but there will be "Error matching Frame Start with Frame End for Virtual Channel x" error reported from iMX6 MIPI_CSI_ERR1 register.

Maxim suggested to use MAX96705 to relace the MAX9271, it can delay the VSYNC invalid time, then the MIPI error will be fixed.

2015-11-17 update: Updated for OV10635 camera support.

File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-11-17.zip

2015-12-04 update:

File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-12-04.zip

Added patch 0009-Max9286-updated-PCLK-edge-setting-for-OV10635.patch to correct the OV10635 PCLK edge setting

2016-03-07 update:

File L3.14.38_GA_MAX9286_Surroundview_Patch_2016-03-07.zip

Added kernel patch for L3.14.38 GA 1.1.0 BSP.

2016-07-26 update:

Files: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.38_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip.

Added gstreamer support.

Added MAX96705 support.

Added patch for L3.14.52_GA1.1.0.

2017-12-11 update:

Added CVBS surround view link: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP

2021-04-26 update:

Some customer reported, when system loading is heavy, sometimes, some camera will flicker left and right. It is caused by SFMC FIFO data lost. The original patch used IDMAC 0 and IDMAC 1 for two cameras on one IPU, this is not the best setting. 

IDMAC 1 is fixed to use 1/4 SMFC FIFO and it will cause IDMAC 0 to use 1/4 SMFC FIFO too. And another 1/2 of SMFC FIFO can't be used in this case.

Some code update to improve it:

For each IPU, please use IDMAC 0 and IDMAC 2 to capture the two cameras.
This needs change the hard coding in "drivers\media\platform\mxc\capture\ipu_csi_enc.c",
"CSI_MEM1" and "IPU_IRQ_CSI1_OUT_EOF" should be changed to "CSI_MEM2" and "IPU_IRQ_CSI2_OUT_EOF". In this case, all SMFC FIFO can be used.

And in "ipu_common.c", function ipu_probe(), the followed code should be changed to make IDMAC2 use high priority too.
/* Set sync refresh channels and CSI->mem channel as high priority */
- ipu_idmac_write(ipu, 0x18800003L, IDMAC_CHA_PRI(0));
+ ipu_idmac_write(ipu, 0x1880000FL, IDMAC_CHA_PRI(0));

[vincent1]

Hi~Li Qiang!

      Thank you for your Patch~

       But our cpu is IMX6Q-Sabresd, Camera is Ov10635. LTIB is L3.0.35_4.1.0

       Can you help me support this Patch ?

       Thanks Sincerely~~

[qiang_li-mpu_se]

We have no plan to back porting the driver to 3.0.35 BSP, please do it by yourself. Thanks!

[farsandy]

你好 Qiang_FSL ，还有颜色问题！

看到了你的邮件，对比了下  发现 邮件中的  ov10635_seting  和  原先代码中的 setting是一模一样的，

还有 MAX9286_OV10635_1280x800_Quad_FSIN_MOD.C 这个文件  ， 不知道对我是否有用，怎么修改原本代码？

[宗標廖]

Hi Qiang Li

Thank you for releasing patch for L3.14.52_GA1.1.0.

I have tested the patch,but  I met  a problem.

The first camera(mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d &) does not show anything,

and after several seconds, error 'VIDIOC_DQBUF failed' happened,all other three cameras also stopped.

Then I tested patch for 3.10.53,all cameras have no problem.

Could you help me?

Thank you!

ZongbiaoLiao

[qiang_li-mpu_se]

You need select MAX96705 or MAX9271 in the kernel config, if it is different with your hardware, it will not work.

[宗標廖]

Hi, Qiang Li

I selected MAX9271.

My problem is three cameras work except the first one.

Thank you!

ZongbiaoLiao

[qiang_li-mpu_se]

What's the error log?

[宗標廖]

Hi Qiang Li

The log is as bellow:

---------------------------------------

root@imx6qsabreauto:/unit_tests# ./max9286.sh

root@imx6qsabreauto:/unit_tests# g_in_width = 1280, g_in_height = 800.

fb_fix.id = DISP3 FG.

g_in_width = 1280, g_in_height = 800.

g_in_width = 1280, g_in_height = 800.

fb: smem_start = 0x72c00000, smem_len = 0xbdd800.

fb_fix.id = DISP3 FG.

fb: frame buffer size = 0x3f4800 bytes.

fb: g_screen_info.xres = 1920, g_screen_info.yres = 1080.

fb: g_display_left = 960.

fb: g_display_top = 0.

fb: g_display_width = 960.

fb: g_display_height = 540.

fb: smem_start = 0x72c00000, smem_len = 0xbdd800.

start time = 1457509993 s, 697691 us

fb: frame buffer size = 0x3f4800 bytes.

fb: g_screen_info.xres = 1920, g_screen_info.yres = 1080.

fb: g_display_left = 0.

fb: g_display_top = 540.

fb_fix.id = DISP3 FG.

fb: g_display_width = 960.

fb: g_display_height = 540.

fb: smem_start = 0x72c00000, smem_len = 0xbdd800.

fb: frame buffer size = 0x3f4800 bytes.

fb: g_screen_info.xres = 1920, g_screen_info.yres = 1080.

fb: g_display_left = 960.

fb: g_display_top = 540.

fb: g_display_width = 960.

fb: g_display_height = 540.

start time = 1457509993 s, 717597 us

start time = 1457509993 s, 717698 us

g_in_width = 720, g_in_height = 624.

fb_fix.id = DISP3 FG.

fb: smem_start = 0x72c00000, smem_len = 0xbdd800.

fb: frame buffer size = 0x3f4800 bytes.

fb: g_screen_info.xres = 1920, g_screen_info.yres = 1080.

fb: g_display_left = 0.

fb: g_display_top = 0.

fb: g_display_width = 960.

fb: g_display_height = 540.

start time = 1457509994 s, 45827 us

ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

VIDIOC_DQBUF failed.

------------------------------------------------------

and what the HDMI display showed is as bellow:

The first camera shows nothing and after several seconds the 'ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0' is output,

all other three cameras also stop and show nothing.

Thank you!

ZongbiaoLiao

[qiang_li-mpu_se]

From your log "g_in_width = 720, g_in_height = 624", I think the IPU0 CSI0 port hasn't been used by the MAX9286, it is still combined with adv7180.

[宗標廖]

Hi Qiang Li

I see.

the adv7180 should be commented out in imx6qdl-sabreauto.dtsi file.

Thank you!

ZongbiaoLiao

[宗標廖]

After commented out adv7180, it became OK.

[liyu1]

你好．．请问你在L3.0.35_4.1.0解决了这个问题吗？　大家一起讨论一下．

[liyu1]

我也需要L3.0.35_4.1.0版本的补丁，谢谢

[haifengjin]

Hi Qiang Li

                 i use de 3.14.52,all patch used,but i have not use,mxc_v4l2_tvin_max9286.tar.gz,in the rootfs，i use unit_tests/mxc_v4l2_tvin.out ,it is nothing to display,can you help me?in the 3.14.52 yocto ,i can not complie the mxc_v4l2_tvin_max9286.tar.gz ,it is something wrong

[qiang_li-mpu_se]

There is binary file in mxc_v4l2_tvin_max9286.tar.gz, you can use it directly.

[haifengjin]

dear Qiang Li：

                      i have use the L3.14.52,it is some error use the binary in mxc_v4l2_tvin_max9286.tar.gz which yours,the log :

./mxc_v4l2_tvin.out: error while loading shared libraries: libg2d.so.0.8: cannot open shared object file: No such file or directory

[qiang_li-mpu_se]

You can link the libg2d.so to libg2d.so.0.8.

[haifengjin]

dear Qiang Li：

                             have you try on the imx6q-sabreauto board？ i can not use the max9286 on sabreauto board，it is some error：

1 -x 0 -g2d &lo:/unit_tests# ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d
[1] 845
root@imx6qdlsolo:/unit_tests# max9286_mipi: sensor number = 1.
mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: mipi_lane_bps = 192 Mbps
mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: value = 0x44.
max9286_mipi 1-006a: ov10635_read_reg:read reg error: reg=0x300a.
max9286_mipi 1-006a: ov10635_check_device: OV10635 hasn't been found, reg[0x300a] = 0x0.
mipi csi2 can not receive sensor clk! MIPI_CSI_PHY_STATE = 0x6f0.
g_in_width = 1280, g_in_height = 800.
fb_fix.id = DISP4 FG.
fb: smem_start = 0x73600000, smem_len = 0x480000.
fb: frame buffer size = 0x180000 bytes.
fb: g_screen_info.xres = 1024, g_screen_info.yres = 768.
fb: g_display_left = 0.
fb: g_display_top = 0.
fb: g_display_width = 960.
fb: g_display_height = 540.

[qiang_li-mpu_se]

Hi Jin, if you supply the camera board with 5V power and using long Coax cable, there is chance to get such issue.

The OV10635 camera board is designed for 7~12V power supply, so you can supply the camera board with 12V power.

[liyu1]

Hi  Qiang li

              你有没有3.0.35的补丁，我们目前给比亚迪开发一款产品，需要用到max9286 ,但以前的ＢＳＰ一直用的3.0.35的内核，能不能提供一下在3.0.35下的max9286的补丁．

　　　　　　　　　　　谢谢！

[qiang_li-mpu_se]

你自己移植一下，应该很容易的。

[liyu1]

我目前是将3.10.53的内核打上补丁之后，对比了改动，，然后在3.0. 35上将这些改动一一加上．不知道这样能不能调通．

[liyu1]

inux_3.10.35/arch/arm/boot/dts/imx6q-sabreauto.dts  中

&mipi_csi {
status = "okay";
lanes = <4>;

mipi-csi2-channel@0 {
ipu_id = <0>;
csi_id = <0>;
v_channel = <0>;
status = "okay";
};

mipi-csi2-channel@1 {
ipu_id = <0>;
csi_id = <1>;
v_channel = <1>;
status = "okay";
};

mipi-csi2-channel@2 {
ipu_id = <1>;
csi_id = <0>;
v_channel = <2>;
status = "okay";
};

mipi-csi2-channel@3 {
ipu_id = <1>;
csi_id = <1>;
v_channel = <3>;
status = "okay";
};
};

这一段在3.0.35中怎么处理？

[qiang_li-mpu_se]

These setting are used in mxc_mipi_csi2.c and mxc_v4l2_capture.c, you can modify the code directly.

By the way, I think you should not compare the patched 3.10.53 files with 3.0.35 files, you can manually modify the 3.0.35 files directly, based on the code modification in patches.

[haifengjin]

hi  qiang li:

                 i use the MAX9286 board,i find the max9286 board supply by 5V, my max9286board supply by maxim,they designed for I.mx6.

[qiang_li-mpu_se]

Hi Haifeng, yes the MAX9286 board was powered from iMX6 board with 5V power, and the OV10635 camera boards can also be powered from iMX6 boards's 5V, but there is another power connector on MAX9286 board which can be used to power the 4xOV10635 camera board, you can connect 12V power to it and select the jump on MAX9286 boards.

So the final power: 5V from iMX6 board to MAX9286 board; 12V from MAX9286 board's power connector to OV10635 camera boards.

[haifengjin]

hi  Li qiang：

                      i find my sensor is AR0140，not OV10635，Where I need to change？thank you

[qiang_li-mpu_se]

Please read the readme file again, it had already told you where to select the camera sensor.

[haifengjin]

hi  Li Qiang：

                    have change the config，But i find i your code only  have AP0101，not have AR0140，i run the board，like this：

[qiang_li-mpu_se]

The camera sensor AR0140 should connect to ISP AP0101, then AP0101 should connected to MAX9271 to output the serial signal.

And I think, without AP0101, the AR0140 can't work in the solution.

[liyu1]

Qiang li:

   在bsp 3.10.53 中 打过补丁如下：

static int mipi_csi2_probe(struct platform_device *pdev)
{

。。。。。。。。。。

ret = of_property_read_u32(np, "lanes", &(gmipi_csi2->lanes));
if (ret) {
dev_err(&pdev->dev, "lanes missing or invalid\n");
goto err;
}
if (gmipi_csi2->lanes > 4) {
dev_err(&pdev->dev, "invalid lanes for mipi csi2!\n");
ret = -EINVAL;
goto err;
}

for_each_child_of_node(np, child) {
struct mipi_csi2_chan *chan;

if (!of_device_is_available(child))
continue;

ret = of_property_read_u32(child, "v_channel", &v_channel);
if (ret) {
dev_err(&pdev->dev, "v_channel missing\n");
goto err;
}

if (v_channel > 3) {
dev_err(&pdev->dev, "v_channel invalid\n");
ret = -EINVAL;
goto err;
}

chan = &(gmipi_csi2->mipi_chan[v_channel]);

ret = of_property_read_u32(child, "ipu_id", &(gmipi_csi2->mipi_chan[v_channel].ipu_id));
if (ret) {
dev_err(&pdev->dev, "ipu_id missing or invalid\n");
goto err;
}

ret = of_property_read_u32(child, "csi_id", &(gmipi_csi2->mipi_chan[v_channel].csi_id));
if (ret) {
dev_err(&pdev->dev, "csi_id missing or invalid\n");
goto err;
}

if ((gmipi_csi2->mipi_chan[v_channel].ipu_id < 0) ||
(gmipi_csi2->mipi_chan[v_channel].ipu_id > 1) ||
(gmipi_csi2->mipi_chan[v_channel].csi_id > 1)) {
dev_err(&pdev->dev, "invalid param for mipi csi2!\n");
ret = -EINVAL;
goto err;
。。。。。。。。。。。。

这部分我在3.0.35中处理是这么处理，麻烦帮我看看有没有问题。

static int mipi_csi2_probe(struct platform_device *pdev)
{
struct mipi_csi2_platform_data *plat_data = pdev->dev.platform_data;
struct resource *res;
u32 mipi_csi2_dphy_ver;
struct mipi_csi2_chan *chan;

int ret , v_channel;

printk("mipi_csi2 probe start !!!!!!!!!!!!!!!!!!!!!\n");
gmipi_csi2 = kmalloc(sizeof(struct mipi_csi2_info), GFP_KERNEL);
if (!gmipi_csi2) {
ret = -ENOMEM;
goto alloc_failed;
}
printk("v_channel is %d !!!!!!!!!!!!!!!!1111\n",v_channel);
/* initialize mutex */
mutex_init(&gmipi_csi2->mutex_lock);

chan = &(gmipi_csi2->mipi_chan[v_channel]);

/* get mipi csi2 informaiton */
gmipi_csi2->pdev = pdev;
gmipi_csi2->mipi_en = false;
//gmipi_csi2->ipu_id = plat_data->ipu_id;
//gmipi_csi2->csi_id = plat_data->csi_id;
//gmipi_csi2->v_channel = plat_data->v_channel;
gmipi_csi2->lanes = plat_data->lanes;
printk("plat_data lanes is %d !!!!!!!!!!!!!!!\n", plat_data->lanes);
v_channel = plat_data->v_channel;
gmipi_csi2->mipi_chan[v_channel].ipu_id= plat_data->ipu_id;
gmipi_csi2->mipi_chan[v_channel].csi_id= plat_data->csi_id;

if ((gmipi_csi2->mipi_chan[v_channel].ipu_id < 0) ||
(gmipi_csi2->mipi_chan[v_channel].ipu_id > 1) ||
(gmipi_csi2->mipi_chan[v_channel].csi_id > 1)) {
dev_err(&pdev->dev, "invalid param for mipi csi2!\n");
ret = -EINVAL;
}

。。。。。。。

   平台数据如下：

  static struct mipi_csi2_platform_data mipi_csi2_pdata = {
.ipu_id = 0, //0 // modify by lee
.csi_id = 0,
.v_channel = 0,
.lanes = 4,
.dphy_clk = "mipi_pllref_clk",
.pixel_clk = "emi_clk",
};

这样操作可以吗？谢谢。在3.0.35上面处理max9286好久了。一直没什么 进展，希望能得到您 的支持。

[liyu1]

            Qiang li:

                            请问你有没有3.0.35 ，　IPU capture drvier to support to four cameras的补丁，有的话，麻烦发给我一下，谢谢

[liyu1]

hi  Qiang li:

                     因为我们跟比亚迪开发的产品当中有设计到tw6865  + max9286 ,,tw6865我已经在３.0.35上调通，但９２８６在３.0.35上调起来比较困难，所以希望能得到您的帮助，

　　　　　　我想第一步应该是使能imx6 4个虚拟通道同时采集数据．　第二部才能调设备，目前max9286有时候能探测到设备AP0101,但有时候又探测不到，不知道是什么原因，怀疑是使能４个虚拟通道出了问题．我是按您发布的3.10.53的补丁移植过来的，希望能得到您的帮助

[zhangwinter]

Hi~Li Qiang!

      Thank you for your Patch~

      I use your patch for my camera OV10635 board,  it is not work. I found  the VSYNC of the OV10635  is discontinuous，the HREF is ok and looks about 30fps.  Can you paste your VSYNC wave here? Thanks

[qiang_li-mpu_se]

Hi Winter, did you measure the correct pin? The HREF is the HSYNC signal, it is for line, so it will not be 30fps.

I think you need check your OV10635 setting, if it is initialized by on board flash ROM, not from the MAX9286 driver, you should update your flash ROM setting. The VSYNC should align with the FSIN signal, this signal is from MAX9271's GPO. One FSIN, one VSYNC.

[timcui]

Hi Qiang，

     The patch of HD 720p sound view we have tested on I.MX6qp-sabreauto .But  now our board designed based on I.mx6qsabresd and BSP version is L3.14.52 for imx6qsabresd. So ,my question is  can you send us the patch for L3.14.52 of  imx6qsabresd ?

[zhangwinter]

Thanks for your reply. There is no flash ROM on OV10635 board.

Follow are the wave of FSIN,HRFE and VSYNC. I think the issure in the OV10635 register setting, My OV10635 is producted in 2014, did you test the patch on the 2014 version chip and have the correct register setting？ 

Coment：yellow is FSIN, blue is VSYNC.

Coment：yellow is HREF, blue is VSYNC.

[dqz]

Hi Li, when i build BSP L3.14.52.1.1,i got some error, please give some help,thanks

bitback fsl-image-gui

[liyu1]

我们的图像在3.0.35版本中，调出来了．

[moowonlee]

I want to get your Test camera board 

    (1) Onsemi AR0140+AP0101+MAX9271 boards.

    (2) OmniVision OV10635+MAX9271 boards.

where can I get them?

Could you please recommend 

[qiang_li-mpu_se]

(1) Onsemi AR0140+AP0101+MAX9271 boards.

We got this board from Onsemi.

(2) OmniVision OV10635+MAX9271 boards.

This board is made by Maxim, you can check with them.

And as I know AVNET China team had also built some camera boards for both AR0140 and OV10635.

"http://entroninc.com/" had also built camera boards for this solution.

[zhanghongliang]

Hi, Qing Li,

我基于imx6-sabresd + L3.14.52调试MAX9286+MAX9271+OV10640（输出分辨率为1280*720），现在是video1、2、3都有图像输出了，但是video0总是提示ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0     VIDIOC_DQBUF failed.我按照上面您给ZongbiaoLiao的修复建议，去掉了占用ipu0的ov5640的驱动和imx6dpl-sabresd.dtb里面相关配置，现在还是存在相同的问题。您帮忙看看问题出在哪里？

我摄像头的分辨率为1280*720，下面是log信息：

max9286_read_reg 0x49 = f.
max9286_mipi: sensor number = 4.
mipi_csi2_set_lanes = 3.
mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: mipi_lane_bps = 594 Mbps
mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: value = 0x2e.
max9271_read_reg 0x07 = 84.
max9286_mipi 2-006a: ov10635_check_device: OV10635 was found.
ovsensor 1 is ok.
mipi_csi2_dphy_status = 0x300.
mipi_csi2_get_error1 = 0x0.
g_in_width = 1280, g_in_height = 720.
fb_fix.id = DISP3 FG.
fb: smem_start = 0x72a00000, smem_len = 0xbdd800.
fb: frame buffer size = 0x3f4800 bytes.
fb: g_screen_info.xres = 1920, g_screen_info.yres = 1080.
fb: g_display_left = 0.
fb: g_display_top = 0.
fb: g_display_width = 960.
fb: g_display_height = 540.
start time = 1482353067 s, 231795 us
ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0
VIDIOC_DQBUF failed.

[qiang_li-mpu_se]

You need check your kernel boot up log to confirm there is no other V4l2 capture driver used video0. You can also use command "ls /dev/video*" to list all video device in your system.

And register IOMUXC_GPR1 (0x020E0004) should also be checked, bit 20 and bit 19 must be 0.

[zhanghongliang]

非常感谢您的回复，bit 19位为1 ，现在已经ok。CSI0_SENS_DATA_FORMAT 配置CSI的数据格式YUYV和UYUV都没起作用，这样修改是否正确，这种问题是哪里导致？

[qiang_li-mpu_se]

The patch had already show you the reference code to set the GPR register in file arch/arm/mach-imx/mach-imx6q.c.

[zhanghongliang]

非常感谢，video0不显示的问题已经搞定了，现在还有个问题，就是通过mxc_v4l2_tvin_max9286历程获取到的YUV数据总是反的，例如,我应用程序中设置g_in_fmt = V4L2_PIX_FMT_UYVY，通过capture_buffers中查看数据实际是YUYV的，如果把g_in_fmt = V4L2_PIX_FMT_YUYV，capture_buffers中查看数据实际又是UYVY，我通过更改CSI0_SENS_DATA_FORMAT 配置CSI的数据格式YUYV和UYUV都没起作用，这样修改是否正确，这种问题是哪里导致？

[zhanghongliang]

补充：我用的sensor是ov490（isp）+ov10640（sensor），sensor输出的数据格式是YUV422---YUYV。

[qiang_li-mpu_se]

Reference to MIPI specification, the YUV422 data transferred on MIPI CSI2 is UYVY, no YUYV, so in max9286_mipi.c, we have the followed code:

max9286_data[0].pix.pixelformat = V4L2_PIX_FMT_UYVY;

if (sensor->pix.pixelformat == V4L2_PIX_FMT_UYVY) {
for (i=0; i<MAX_SENSOR_NUM; i++)
mipi_csi2_set_datatype(mipi_csi2_info, i, MIPI_DT_YUV422);

If your camera can only be YUYV format, you should change the max9286_mipi.c as followed:

max9286_data[0].pix.pixelformat = V4L2_PIX_FMT_YUYV;

if (sensor->pix.pixelformat == V4L2_PIX_FMT_YUYV) {
for (i=0; i<MAX_SENSOR_NUM; i++)
mipi_csi2_set_datatype(mipi_csi2_info, i, MIPI_DT_YUV422);

Note: MIPI_DT_YUV422 can't be changed, it is defined by MIPI CSI2 specification for UYVY, there is no MIPI data type for YUYV.

[zhanghongliang]

Thanks. The problem has been resoved.

[zhanghongliang]

我想把四路摄像头，按上下左右格式压缩成一路720p的视频进行存储，现在四路视频整合要怎么实现？有没有什么建议？