iMX6DQ MAX9286 MIPI CSI2 720P camera surround view solution for Linux BSP

[#1]

For iMX6DQ, there are two IPUs, so they can support up to 4 cameras at the same time. But the default BSP can only support up to two cameras at the same time.

The attached patch can make the BSP support up to 4 cameras based on 3.10.53 GA 1.1.0 BSP.

The 4 cameras can be:

- 1xCSI, 3xMIPI

- 2xCSI, 2xMIPI

- 4xMIPI

For 4xMIPI case, the four cameras should be combined on the single MIPI CSI2 interface, and each camera data should be transfered on a mipi virtual channel.

In this patch, we given the example driver for Maxim MAX9286, it was verified working on iMX6DQ SabreAuto board. The input to MAX9286 is four 720P30 cameras.

The verified camera boards:

    (1) Onsemi AR0140+AP0101+MAX9271 boards.

    (2) OmniVision OV10635+MAX9271 boards.

The MIPI CSI2 CVBS camera surround view solution can be found at: iMX6DQ ISL79985/79987 MIPI CSI2 CVBS camera surround view solution for Linux BSP

The MIPI CSI2 CVBS HD camera surround view solution can be found at: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP

The kernel patches:
0001-IPU-update-IPU-capture-driver-to-support-up-to-four-.patch
     Updated IPU common code to support up to four cameras.

0002-Add-Max9286-support-on-SabreAuto-board-which-can-sup.patch
     MAX9286 driver, it includes MAX9271, AP0101 and AR0140 drivers.

0003-Remove-the-page-size-align-requirement-for-v4l2-capt.patch
     With this patch, the mxc_v4l2_tvin test application can use overlay framebuffer as V4l2 capture buffer directly.

0004-Max9286-skip-AP0101-camera-re-initialization.patch
     If the camera board's power had been kept after initialized, this patch will bypass the re-initialization to reduce the start up time.

0005-Max9286-set-I2C-speed-to-400Kbps.patch
    Set I2C to 400Kbps to reduce the AP0101+AR0140 initialization time.

0006-Max9286-add-retry-for-MAX9271-I2C-access.patch
    Added retry for MAX9271 I2C access.

0007-Max9286-Add-support-for-OV10635-camera.patch
    Updated code for OV10635 camera.

0008-Max9286-support-auto-detect-camera-number.patch
    Make the Max9286 driver can detect the camera number automatically.

How to builld the kernel with MAX9286 support:

      make imx_v7_defconfig

      make menuconfig (In this command, you should select the MAX9286 driver:

            Device Drivers  --->

                  <*> Multimedia support  --->

                        [*]   V4L platform devices  --->

                              <*>   MXC Video For Linux Video Capture

                                      MXC Camera/V4L2 PRP Features support  --->

                                          <*>Maxim max9286 GMSL Deserializer Input support

                                              Select Camera Sensor (OmniVision OV10635 camera sensor)  // Or (Onsemi AP0101 and AR0140 camera sensor)

                                          <*>mxc VADC support

                                          <*>Select Overlay Rounting (Queue ipu device for overlay library)

                                          <*>Pre-processor Encoder library

                                          <*>IPU CSI Encoder library)

      make zImage

      make dtbs

The built out image file:

      arch/arm/boot/dts/imx6q-sabreauto.dtb

      arch/arm/boot/zImage

"mxc_v4l2_tvin_max9286.tar.gz" is the test application, test command to capture the four cameras and render on 1080P HDMI display:

/mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d &

/mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d &

/mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d &

/mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &

Some hardware check point on AR0140+AP0101+MAX9271 camera board (Please get MAX9286 and OV10635 schematics from Maxim):

1. In this patch, MAX9286's I2C address is 0x4D, so ADD0 and ADD1 should be connected to high.

AP0101's I2C address is 0xBA, so SADDR should be connected to high.

2. AP0101's DOUT0~DOUT7 should be connected to MAX9271's DIN7~DIN0, the order should be switched, MSB connected to LSB.

3. MAX9271's GPO pin should be connected to AP0101's FRAME_SYNC pin. The pull down resistance on FRAME_SYNC pin should not be 0 ohm.

Some known limitation:

1. AP0101's VSYNC invalid time, last video line's HSYNC to VSYNC porch's max value is 255 pixel clocks, it is not enough for MAX9286 to generate the Frame End MIPI packets for each camera. So in order to let iMX6DQ to capture 1280x720 video for each camera, we had let AP0101 output 1280*724 frame size, and iMX6 will only capture 720 lines, the remained video data and Frame End will be ignored.

This solution will not impact the function, but there will be "Error matching Frame Start with Frame End for Virtual Channel x" error reported from iMX6 MIPI_CSI_ERR1 register.

Maxim suggested to use MAX96705 to relace the MAX9271, it can delay the VSYNC invalid time, then the MIPI error will be fixed.

2015-11-17 update: Updated for OV10635 camera support.

File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-11-17.zip

2015-12-04 update:

File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-12-04.zip

Added patch 0009-Max9286-updated-PCLK-edge-setting-for-OV10635.patch to correct the OV10635 PCLK edge setting

2016-03-07 update:

File L3.14.38_GA_MAX9286_Surroundview_Patch_2016-03-07.zip

Added kernel patch for L3.14.38 GA 1.1.0 BSP.

2016-07-26 update:

Files: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.38_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip.

Added gstreamer support.

Added MAX96705 support.

Added patch for L3.14.52_GA1.1.0.

2017-12-11 update:

Added CVBS surround view link: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP

2021-04-26 update:

Some customer reported, when system loading is heavy, sometimes, some camera will flicker left and right. It is caused by SFMC FIFO data lost. The original patch used IDMAC 0 and IDMAC 1 for two cameras on one IPU, this is not the best setting. 

IDMAC 1 is fixed to use 1/4 SMFC FIFO and it will cause IDMAC 0 to use 1/4 SMFC FIFO too. And another 1/2 of SMFC FIFO can't be used in this case.

Some code update to improve it:

For each IPU, please use IDMAC 0 and IDMAC 2 to capture the two cameras.
This needs change the hard coding in "drivers\media\platform\mxc\capture\ipu_csi_enc.c",
"CSI_MEM1" and "IPU_IRQ_CSI1_OUT_EOF" should be changed to "CSI_MEM2" and "IPU_IRQ_CSI2_OUT_EOF". In this case, all SMFC FIFO can be used.

And in "ipu_common.c", function ipu_probe(), the followed code should be changed to make IDMAC2 use high priority too.
/* Set sync refresh channels and CSI->mem channel as high priority */
- ipu_idmac_write(ipu, 0x18800003L, IDMAC_CHA_PRI(0));
+ ipu_idmac_write(ipu, 0x1880000FL, IDMAC_CHA_PRI(0));

[2479420618]

Thinks for your help!

I have other problem with devicetree.Look at the picture,reg=6A,but max9286 datasheet is no this address.

[2479420618]

Hi Qiang Li,

  My problem is as bellow.Can you help me to deal with it.

[wang007987]

Hi， Qiang Li：

        在max9286_mipi.c中我发现如下内容：

reg_item2_t ap0101_settings[] = {
//Step2-PLL_Timing - YUV8 720p30
{0xCA84, 8, 0x05}, // CAM_SYSCTL_PLL_CONTROL
{0xCA88, 16, 0x0462}, // CAM_SYSCTL_PLL_DIVIDER_M_N_1_CLK
{0xCA8A, 16, 0x0462}, // CAM_SYSCTL_PLL_DIVIDER_M_N_2_CLK
{0xCA8C, 16, 0x00FF}, // CAM_SYSCTL_PLL_DIVIDER_P_1_CLK
{0xCA8E, 16, 0x007F}, // CAM_SYSCTL_PLL_DIVIDER_P_2_CLK
{0xCA9C, 16, 0x0405}, // CAM_PORT_PARALLEL_CONTROL, YUV8
{0xC838, 32, 0x080210C5}, // CAM_SENSOR_CONTROL_EXTERNAL_PLL
{0xC804, 16, 0x000C}, // CAM_SENSOR_CFG_Y_ADDR_START

 ................................

我并未发现0xCA84之类addr的来源 ，是否可以告知，非常感谢！

[qiang_li-mpu_se]

You can check with the AP0101 camera chip vendor.

[ferhatolgun]

Hi Qiang Li,

In our project we have one camera connected to max9286 and we applied your patches to 4.1.15 kernel; however when we test it, we get the following picture:

Logs are as follow:

root@esomimx6q:~# ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 400 -oh 240 -d 1 -x 0
[ 3144.243857] max9286_mipi: sensor number = 1.
[ 3144.248939] mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: mipi_lane_bps = 148 Mbps
[ 3144.257446] mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: value = 0x42.
[ 3144.743416] max9286_mipi 1-006a: ap0101_initialize: AP0101 was found, index = 1.
[ 3144.751541] max9286_mipi 1-006a: ap0101_initialize:AP0101 is already in Slave (Shutter-Sync) Mode.
g_in_width = 1280, g_in_height = 720.
fb_fix.id = DISP4 FG.
fb: smem_start = 0x6a100000, smem_len = 0x8ca00.
fb: frame buffer size = 0x2ee00 bytes.
fb: g_screen_info.xres = 400, g_screen_info.yres = 240.
fb: g_display_left = 0.
fb: g_display_top = 0.
fb: g_display_width = 400.
fb: g_display_height = 240.
start time = 1535715437 s, 436911 us
[ 3154.993386] ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0
VIDIOC_DQBUF failed.

We could not understand the reason of the problem (also we adjusted gpr1 pins 19 and 20 as 0). 

Secondly, the following logs show us that there are problems with /dev/video 1 and dev/video3 about csi channel although dev/video0 and /dev/video2 is ok , however all of them have timeout error (note that since only one camera is connected, only "cat /dev/video0" command can initialize ap0101).

root@esomimx6q:~# cat /dev/video0
[ 3313.913877] max9286_mipi: sensor number = 1.
[ 3313.918963] mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: mipi_lane_bps = 148 Mbps
[ 3313.926941] mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: value = 0x42.
[ 3314.413402] max9286_mipi 1-006a: ap0101_initialize: AP0101 was found, index = 1.
[ 3314.421469] max9286_mipi 1-006a: ap0101_initialize:AP0101 is already in Slave (Shutter-Sync) Mode.
[ 3324.583392] ERROR: v4l2 capture: mxc_v4l_read timeout counter 0
cat: read error: Timer expired
root@esomimx6q:~# cat /dev/video1
[ 3337.083393] ERROR: v4l2 capture: mxc_v4l_read timeout counter 0
[ 3337.089359] imx-ipuv3 2400000.ipu: Not a CSI channel
cat: read error: Timer expired
root@esomimx6q:~# cat /dev/video2
[ 3351.493394] ERROR: v4l2 capture: mxc_v4l_read timeout counter 0
cat: read error: Timer expired
root@esomimx6q:~# cat /dev/video3
[ 3403.463393] ERROR: v4l2 capture: mxc_v4l_read timeout counter 0
[ 3403.469360] imx-ipuv3 2800000.ipu: Not a CSI channel
cat: read error: Timer expired

Can you help us with them? What is the reason of "not a csi channel error" on video1 and video3 while others don't give this error. Also the first error is more important, which is the error of "vidioc_dqbuf failed".

Thank you

Best regards

[qiang_li-mpu_se]

For error "not a csi channel error", I think you haven't ported the patch successfully.

For error "vidioc_dqbuf failed", that means no available video frames had been captured. Maybe you need check your AP0101 camera setting.

[ferhatolgun]

Hi again Qiang Li,

As you can see in our logs, our camera is in shutter-sync mode, thus doesn't this means our camera setting is correct? What do you mean by camera setting, can you be more specific and explain it in detail?. Also MIPI_CSI_ERR1 register is 0, there is no error of transmission of bits, clock signal etc and mpi csi2 can receive the data correctly. Doesnot that mean that camera sends correct signals and is configured properly? 

Secondly, we reviewed all patches and applied them directly by copy-paste, also we double checked whether there is an error; thus, we are (almost) sure that we ported correctly. What is the key points we need to look at? Can there be any other possible cause of this problem?

Thank you,

Best regards

[qiang_li-mpu_se]

“ap0101_initialize:AP0101 is already in Slave (Shutter-Sync) Mode”

This log means your camera had already be initialized, so you need check if there is flash ROM on your camera module which will initialize it during power on, if so, you need update that flash ROM to use the driver's setting.

[ferhatolgun]

Dear Qiang Li, we removed eeprom on the camera module and we switched it to host configuration mode by setting SPI-SDI pin to LOW. Thus it is initialized successfully by the driver of max9286 in the beginning, after kernel is loaded. Do you have any other suggestion for both "not a csi channel" and vidioc_dqbuf errors?

Thank you,

Best regards

[qiang_li-mpu_se]

That means your patch porting has error, please check it.

[ferhatolgun]

Dear Qiang Li,

I would like to know the reason of your hardware point stated as below.

"2) AP0101's DOUT0~DOUT7 should be connected to MAX9271's DIN7~DIN0, the order should be switched, MSB connected to LSB"

In our reference camera board supplied by Onsemi (MARS module) these pins are connected directly (DOUT0-DOUT7 to DIN0-DIN7). Is there any way to switch these connection by software? I could not find on datasheets of MAX96705 and AP0101.

Thanks 

[dokim]

Hi, Qiang Li

May I know the reason for i.MX8MQ can't support multiple virtual channel?
According to the features and functional description of MIPI-CSI2 Controller on Reference Manual for i.MX8M Quad,

i.MX8M seems to support multiple virtual channel.(13.6.1.1 and 13.6.3)

Thanks.

[qiang_li-mpu_se]

Hi Dongoh,

The iMX8MQ can receive video with multiple virtual channels, but there is no hardware module to split the video from each virtual channel.

That means you need use software to split, and the MIPI sender need insert some flag to identify the data for each virtual channel.

[qiang_li-mpu_se]

Hi Ferhat,

MAX9286's data mapping needs such MSB to LSB switch, if you are using MAX96705, it can do such swap by setting, but MAX9271 doesn't support it. And AP0101 doesn't dupport such swap too, so hardware should be modified.

[ferhatolgun]

Hi Qiang,

Does the patch you provided include such swap settings on MAX96705? We use MAX96705 and do not do such swap by driver

Thank you

[qiang_li-mpu_se]

Hi Ferhat，

Yes, the switch code:

 max9271_write_reg(0, 0x20, 0x09);
 max9271_write_reg(0, 0x21, 0x08);
 max9271_write_reg(0, 0x22, 0x07);
 max9271_write_reg(0, 0x23, 0x06);
 max9271_write_reg(0, 0x24, 0x05);
 max9271_write_reg(0, 0x25, 0x04);
 max9271_write_reg(0, 0x26, 0x03);
 max9271_write_reg(0, 0x27, 0x02);
 max9271_write_reg(0, 0x28, 0x01);
 max9271_write_reg(0, 0x29, 0x00);

 max9271_write_reg(0, 0x30, 0x19);
 max9271_write_reg(0, 0x31, 0x18);
 max9271_write_reg(0, 0x32, 0x17);
 max9271_write_reg(0, 0x33, 0x16);
 max9271_write_reg(0, 0x34, 0x15);
 max9271_write_reg(0, 0x35, 0x14);
 max9271_write_reg(0, 0x36, 0x13);
 max9271_write_reg(0, 0x37, 0x12);
 max9271_write_reg(0, 0x38, 0x11);
 max9271_write_reg(0, 0x39, 0x10);

[dokim]

Thanks for your ans.

Then, is there any example code for split the video from each virtual channel?
If not, may I know how to read the raw packet data of mipi_csi?

Thanks.

[gaozhihe]

The "0003-Add-Max9286-support-on-SabreAuto-board-which-can-sup.patch" files of the L3.14.38_GA andL3.14.52_GA patches have errors.

+static int max9271_write_reg(int index, u8 reg, u8 val)
+{
+ s32 ret;
+ int retry, timeout = 10;
+
+ max9286_data[0].i2c_client->addr = ADDR_MAX9271 + index;
+ for (retry = 0; retry < timeout; retry ++) {
+ ret = i2c_smbus_write_byte_data(max9286_data[0].i2c_client, reg, val);
+ if (val < 0)
+ msleep(5);
+ else
+ break;
+ }
+
+ if (retry >= timeout) {
+ dev_info(&max9286_data[0].i2c_client->dev,
+ "%s:write reg error:reg=%2x,val=%2x\n", __func__,
+ reg, val);
+ return -1;
+ }
+
+ return 0;
+}

[qiang_li-mpu_se]

Hi DONGON KIM,

Some video combining chip can insert some bytes into each video line, then software can check these inserted data to identify the camera number.

[ferhatolgun]

Hi Qiang Li,

Could not find the switch code on patch files. Could you please update the links with your latest versions?

Thanks and best regards

[qiang_li-mpu_se]

Hi Ferhat,

The camera I used doesn't need such switch, so no such code in patch, but I had show you the sample code, for detail setting on MAX96705 switch, you can check with Maxim.

[lemficb]

您好！我们目前在调一款IMX225+FH8536+MAX96705的摄像头，平台：IMX6+MAX9286，Kernel版本3.14.52，视频基本轮廓已经正常，但色彩不太正常，如图

为了验证是否前端ISP输出有问题，我们做了实验，断开ISP与MAX96705的连接，ISP输出端飞线到一个decoder(TW8836)，然后通过调试8836，发现可以正常输出图像，即ISP输出的数据没有问题，且输出格式为YVYU。

那么我想请教一下，max96705是否可以把YVYU的数据转成UYVY的格式再传给MAX9286？或者在imx6端应该怎么处理显示部分？

[qiang_li-mpu_se]

这个应该是camera到MAX96705的高低位数据线反了，设置96705的时候可以反转(斜体部分):

#ifdef MAX96705
 //Invert VSYNC
 max9271_write_reg(0, 0x40, 0x2F);
 msleep(2);
 max9271_write_reg(0, 0x08, 0x81);
 msleep(2);

 max9271_write_reg(0, 0x97, 0x5F);

 max9271_write_reg(0, 0x20, 0x09);
 max9271_write_reg(0, 0x21, 0x08);
 max9271_write_reg(0, 0x22, 0x07);
 max9271_write_reg(0, 0x23, 0x06);
 max9271_write_reg(0, 0x24, 0x05);
 max9271_write_reg(0, 0x25, 0x04);
 max9271_write_reg(0, 0x26, 0x03);
 max9271_write_reg(0, 0x27, 0x02);
 max9271_write_reg(0, 0x28, 0x01);
 max9271_write_reg(0, 0x29, 0x00);

 max9271_write_reg(0, 0x30, 0x19);
 max9271_write_reg(0, 0x31, 0x18);
 max9271_write_reg(0, 0x32, 0x17);
 max9271_write_reg(0, 0x33, 0x16);
 max9271_write_reg(0, 0x34, 0x15);
 max9271_write_reg(0, 0x35, 0x14);
 max9271_write_reg(0, 0x36, 0x13);
 max9271_write_reg(0, 0x37, 0x12);
 max9271_write_reg(0, 0x38, 0x11);
 max9271_write_reg(0, 0x39, 0x10);
#else

[ghostar]

@Qiang Li - Mpu Se

您好！感谢回复！

         首先，camera到MAX96705的高低位数据线确实是反的，但我们只接了8根数据线而不是10根，如上图所示的画面已经是把数据线反过来后的图像，具体的设置如下：

max9271_write_reg(i, 0x20, 0x07);
max9271_write_reg(i, 0x21, 0x06);
max9271_write_reg(i, 0x22, 0x05);
max9271_write_reg(i, 0x23, 0x04);
max9271_write_reg(i, 0x24, 0x03);
max9271_write_reg(i, 0x25, 0x02);
max9271_write_reg(i, 0x26, 0x01);
max9271_write_reg(i, 0x27, 0x00);

max9271_write_reg(i, 0x30, 0x17);
max9271_write_reg(i, 0x31, 0x16);
max9271_write_reg(i, 0x32, 0x15);
max9271_write_reg(i, 0x33, 0x14);
max9271_write_reg(i, 0x34, 0x13);
max9271_write_reg(i, 0x35, 0x12);
max9271_write_reg(i, 0x36, 0x11);
max9271_write_reg(i, 0x37, 0x10);

然后后面我们按您的设置改了一下，改后画面丢失的轮廓细节更多了。

这还可能是别的什么因素造成的吗？感谢！

[qiang_li-mpu_se]

目前MAX96705的设置都是按照10根数据线的方法来采样camera的图像，如果只连了8根的话，反转后，你们的高2位图像就丢失了，基本上是不行的。你们可以咨询一下Maxim，8根的方式如何处理。

[志锋杨]

ferhatolgun‌你好，请问你的问题解决了吗，我这边现在也遇到跟你一样的问题，解决了的话，能否告知，谢谢！

[jiangdezhi]

要看一下你的输入是：

raw data 输入还是YUV输入，如果是raw cmos sensor 输入应该是12bit的。丢位会出问题。如果是8位YUV输入（BT656）,那要看你是不是位对上了，0～7 or 2～9.。。。。serdis 通常做个传输，输入是啥，输出也是啥～

dezhijiang@126.com

[志锋杨]

Hi，我们现在的板子是IMX6QP+MAX9286+MAX96705+AP0101AT+AR0143AT，kernel版本是L4.1.15。

AP0101上挂了个flash，用的供应商提供的固件，视频格式为：YUV422 8bit/LSB，已经确认ISP输出的视频数据正常

目前只连接了一个摄像头模组进行调试，打上您提供的patch之后，出现了下面的提示：

root@imx6qp-avm718:/unit_tests# ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 0 -x 0

In MVC: mxc_v4l_open
device name is Mxc Camera
End of mxc_v4l_open: v2f pix widthxheight 288 x 352
End of mxc_v4l_open: crop_bounds widthxheight 1280 x 720
End of mxc_v4l_open: crop_defrect widthxheight 1280 x 720
End of mxc_v4l_open: crop_current widthxheight 1280 x 720
On Open: Input to ipu size is 1280 x 720
max9286_mipi: sensor number = 1.
mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: mipi_lane_bps = 148 Mbps
mxc_mipi_csi2 21dc000.mipi_csi: mipi_csi2_reset: value = 0x42.
max9286_mipi 1-006a: ap0101_initialize: AP0101 was found, index = 1.
max9286_mipi 1-006a: ap0101_initialize:AP0101 is already in Slave (Shutter-Sync) Mode.
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl 80685600
case VIDIOC_QUERYCAP
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0045627
case VIDIOC_S_INPUT
In MVC:mxc_streamoff
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c02c563a
case VIDIOC_CROPCAP
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0cc5616
case VIDIOC_S_PARM
In mxc_v4l2_s_param
Current capabilities are 0
Current capturemode is 0 change to 0
Current framerate is 30 change to 0
clock_curr=mclk=27000000
g_fmt_cap returns widthxheight of input as 1280 x 720
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0cc5605
case VIDIOC_S_FMT
In MVC: mxc_v4l2_s_fmt
type=V4L2_BUF_TYPE_VIDEO_CAPTURE
End of mxc_v4l2_s_fmt: v2f pix widthxheight 1280 x 720
End of mxc_v4l2_s_fmt: crop_bounds widthxheight 1280 x 720
End of mxc_v4l2_s_fmt: crop_defrect widthxheight 1280 x 720
End of mxc_v4l2_s_fmt: crop_current widthxheight 1280 x 720
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0cc5604
case VIDIOC_G_FMT
In MVC: mxc_v4l2_g_fmt type=1
type is V4L2_BUF_TYPE_VIDEO_CAPTURE
End of mxc_v4l2_g_fmt: v2f pix widthxheight 1280 x 720
End of mxc_v4l2_g_fmt: crop_bounds widthxheight 1280 x 720
End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1280 x 720
End of mxc_v4l2_g_fmt: crop_current widthxheight 1280 x 720
g_in_width = 1280, g_in_height = In MVC:mxc_v4l_ioctl
720.
In MVC: mxc_v4l_do_ioctl c0145608
case VIDIOC_REQBUFS
In MVC:mxc_streamoff
MVC: In mxc_free_frame_buf
In MVC:mxc_allocate_frame_buf - size=1843200
fb_fix.id = DISP3 BG - DI1.
It is background screen, only full screen default format was supported.
fb: smem_start = 0x73500000, smemIn MVC:mxc_v4l_ioctl
_len = 0xbdd800.
fb: frame buffIn MVC: mxc_v4l_do_ioctl c0445609
er size = 0x3f4800 bytes.
fb: g case VIDIOC_QUERYBUF
_screen_info.xres = 1920, g_screeIn MVC:mxc_v4l2_buffer_status
n_info.yres = 1080.
fb: g_displIn MVC:mxc_mmap
ay_left = 0.
fb: g_display_top pgoff=0x72d00, start=0x75e00000, end=0x75fc2000
= 0.
fb: g_display_width = 1920In MVC:mxc_v4l_ioctl
.
In MVC: mxc_v4l_do_ioctl c0445609

case VIDIOC_QUERYBUF
In MVC:mxc_v4l2_buffer_status
In MVC:mxc_mmap
pgoff=0x72f00, start=0x75c3e000, end=0x75e00000
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0445609
case VIDIOC_QUERYBUF
In MVC:mxc_v4l2_buffer_status
In MVC:mxc_mmap
pgoff=0x73100, start=0x75a7c000, end=0x75c3e000
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c0445609
case VIDIOC_QUERYBUF
In MVC:mxc_v4l2_buffer_status
In MVC:mxc_mmap
pgoff=0x73300, start=0x758ba000, end=0x75a7c000
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl c044560f
case VIDIOC_QBUF
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl 40045612
case VIDIOC_STREAMON
In MVC:mxc_streamon
IPU:In csi_enc_enabling_tasks
In csi_enc_setup
start time = 1540834851 s, 23859 In MVC:mxc_v4l_ioctl
us
In MVC: mxc_v4l_do_ioctl c0445611
case VIDIOC_DQBUF
In MVC:mxc_v4l_dqueue
ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0
VIDIOC_DQBUF failed.
In MVC:mxc_v4l_ioctl
In MVC: mxc_v4l_do_ioctl 40045613
case VIDIOC_STREAMOFF
In MVC:mxc_streamoff
CSI_MEM0:fffffc0,CIS_MEM1:10ffffc1,CSI_MEM2:11ffffc2,CSI_MEM3:12ffffc3,CSI_PRP_ENC_MEM:13ffffd4,CSI_PRP_VF_MEM:14ffffd5
channel:fffffc0
In MVC:mxc_free_frames
In MVC:mxc_v4l_close
In MVC:mxc_streamoff
mxc_v4l_close: release resource
MVC: In mxc_free_frame_buf
In MVC:mxc_free_frames

看起来是视频数据没拿到，我这边测量了MAX9286 mipi的输出，其中mipi clk是72MHz，数据pin脚上的波形如下图；

我考虑了AP0101与MAX96705 数据脚错位或者反接的问题，但从之前别人的情况来看，数据脚错位的话，是能够看到视频的，只是视频画面异常，并不会出现VIDIOC_DQBUF failed；

请问您这边能不能提供调试的方向，谢谢！

[志锋杨]

Hi，我们镜头用的MAX96705+AP0101+AR0143AT，AP0101上挂了个flash，ISP输出是YUV422/8bit/LSB，目前是出现了VIDIOC_DQBUF failed的提示，看了之前别人的问题，如果是data错位的问题，好像是不会出现VIDIOC_DQBUF failed的

[ferhatolgun]

Hi,

VIDIOC_DQBUF  error dissappears when I changed the test hardware with a new set. I will work on the real reason but it seems to be a hardware issue. Although I sometimes  get VIDIOC_DQBUF and MIPI CLK errors; I can succesfully get the image with the below configuration.

• AR0140 sensor
• AP0101 ISP
• MAX96705 Serializer
• MAX9286 Deserializer
• Our designed daughter board between MAX9286 EVM and iMX6 board.

We use kernel 4.1.15 and use below switch code:

+ max9271_write_reg(i, 0x20, 0x07);
+ max9271_write_reg(i, 0x21, 0x06);
+ max9271_write_reg(i, 0x22, 0x05);
+ max9271_write_reg(i, 0x23, 0x04);
+ max9271_write_reg(i, 0x24, 0x03);
+ max9271_write_reg(i, 0x25, 0x02);
+ max9271_write_reg(i, 0x26, 0x01);
+ max9271_write_reg(i, 0x27, 0x00);
+
+
+ max9271_write_reg(i, 0x30, 0x17);
+ max9271_write_reg(i, 0x31, 0x16);
+ max9271_write_reg(i, 0x32, 0x15);
+ max9271_write_reg(i, 0x33, 0x14);
+ max9271_write_reg(i, 0x34, 0x13);
+ max9271_write_reg(i, 0x35, 0x12);
+ max9271_write_reg(i, 0x36, 0x11);
+ max9271_write_reg(i, 0x37, 0x10);

Thanks and best regards

[qiang_li-mpu_se]

The FLASH ROM setting on camera module should follow the driver's setting.

[志锋杨]

你好，现在画面出来了，但是画面异常，画面整体绿色，灯光部分是粉红色，猜测是因为Sensor输出是YUYV导致，我在应用程序mxc_v4l2_tvin.c里面，将捕获到的数据将其转成UYVY后进行显示，画面颜色正常；验证了这个问题是sensor输出格式为YUYV导致；

由于在应用程序中将数据转成UYVY的话，占用系统资源，所以，在驱动中做以下修改，在max9286_mipi.c中，做以下修改：

- max9286_data[0].pix.pixelformat = V4L2_PIX_FMT_UYVY; 

+ max9286_data[0].pix.pixelformat = V4L2_PIX_FMT_YUYV; 

- if(sensor->pix.pixelformat == V4L2_PIX_FMT_UYVY){

+ if(sensor->pix.pixelformat == V4L2_PIX_FMT_YUYV){

      for(i = 0; i < MAX_SENSOR_NUM;i++)

            mipi_csi2_set_datatype(mipi_csi2_info,i,MIPI_DT_YUV422);

}

然后，执行以下命令:

./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -if YUYV

跟踪查看CSI_SENS_CONF寄存器值为0x900,就是说SENS_DATA_FORMAT已经成功配置为YUYV；

但是，HDMI输出的画面仍然异常，跟未修改之前的画面一模一样，感觉就像是修改未生效；

请问，上述修改方式是否有问题，有没有别的地方需要同步修改，谢谢！

[志锋杨]

Hi，我将VS极性反了一下，然后再加上您提供的代码后，现在已经有画面出来了，感谢！

[qiang_li-mpu_se]

按照MIPI的规范，MIPI CSI2传输的YUV422应该是UYVY格式的，但是你们传的是YUYV，所以就不能用一个统一的设置去处理。

mxc_v4l2_tvin里面，g_in_fmt（g_g2d_fmt）的设置默认会设到Driver里面去，然后render的时候也会用到，针对你们的情况，Driver里面应该全部用UYVY的设置，然后要做render的时候，把g_in_fmt（g_g2d_fmt）改成YUYV就可以了。

[志锋杨]

Hi，之前我的说法有误，真实原因是CSI2IPU gasket输出格式被配置为YUYV，将CSI2IPU_SW_RST寄存器的YUV422_8BIT_FM设置为UYVY后问题解决，谢谢！

[志锋杨]

Hi pengtao，我们现在镜头用的跟你一样的方案，AR0143+AP0101+MAX96705+MAX9286+IMX6Q，一路的话可以正常显示，但是接入两路的话，9286的MIPI无输出，读取9286的寄存器，说是没有同步上，我们测了两个镜头模组中AP0101的VS_OUT信号，发现确认没同步，请问您是怎么解决的，谢谢！

[qiang_li-mpu_se]

camera模组的工作模式，需要参考驱动里面的设置，设成SYNC模式，默认带有Flash ROM的camera模组，flash里面的模式一般都是Free run模式，所以多个camera是不能同步的。

[志锋杨]

Hi，Qiang Li，谢谢回复，我们现在确实是镜头模组上挂了个flash，不过flash里面的固件是安森美的代理商提供的，他们确认已经工作在framesync模式，并且这个固件已经有别的客户在量产，我们读出AP0101的0xC88C，0xC88D，0xC88E寄存器，值分别是0x03,0x03,0x00，同时通过逻辑分析仪分析AP0101对AR0143的配置，感觉也应该是正确的；max9286这端已经确认生成的framesync信号有传输到AP0101，现在sensor出来的帧率是25fps；

请问针对25fps，max9286这边的配置应该如果修改，盼复，谢谢！

[qiang_li-mpu_se]

MAX9286这边，你可以试试修改寄存器 0x08

max9286_write_reg(0x08, 0x26);

改成

max9286_write_reg(0x08, 0x2C);

你们Camera模组里面Flash的设置最好要参考Driver里面的设置，FrameSync模式下面的帧率，应该是由MAX9286这边决定的，所以理论上不应该出现MAX9286按照30fps发送FrameSync给camera，而camera是按照25fps送出图像的现象。

[balasubramaniyam]

Hi,

How to build the "Surround View Application" and its required packages in Yocto for i.MX6 QuadPLUS

Hoping for a better guidance.

Regards,

Bala

[765077233]

Hi Qiang Li,

I meet following problem:when I use command without -g2d argument,it display following error,

root@imx6qsabresd:/# ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 1280 -oh 720 -d 1 -x 0
g_in_width = 1280, g_in_height = 720.
fb_fix.id = DISP3 FG.
fb: smem_start = 0x26700000, smem_len = 0xa8c000.
fb: frame buffer size = 0x1c2000 bytes.
fb: g_screen_info.xres = 1280, g_screen_info.yres = 720.
fb: g_display_left = 0.
fb: g_display_top = 0.
fb: g_display_width = 1280.
fb: g_display_height = 720.
IPU_CHECK_TASK failed.

But when I use the command ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 1280 -oh 719 -d 1 -x 0,the display image is OK.

what's the impossible reason?Thanks!

[qiang_li-mpu_se]

The default IPU task doesn't support memory copy use case, input resolution and format is same as output.

[765077233]

Thans for your support!

1. You means that if input resolution and format is same as output,IPU will not deal with it. Right?

2. I still have another question,when I use command with -g2d argument,like this:

root@imx6qsabresd:/# ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 1280 -oh 720 -d 1 -x 0 -g2d，screen tearing occurs like the following picture:

Could you  give me some advice?Thanks!

[yafengge126]

Hi，Liqiang

Qiang_FSL‌

We are developing 4 camera surroundview application, based on imx6d/OV10635/MAX96705/MAX9286 boards,

the patch we apply is L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip. 

but the image color is NOT right as following:

Can you please help ?

[yafengge126]

Qiang_FSL‌

补充，我这里的环境如下：

• OV10635 sensor
• MAX96705 Serializer
• MAX9286 Deserializer
• kernel 4.1.15
• patch L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip

图像可以显示，就是颜色不对。

[qiang_li-mpu_se]

The current mxc_v4l2_tvin demo application will render camera video to current frame buffer directly, so there will be tearing issue.

For real use case, you need use multi-buffers, for example, the virtual framebuffer size is 3 times of real display buffer, and each time only one buffer is showing on display, so when buffer 1 is showing on display, you can draw combined camera videos into buffer 2, after finished drawing, call ioctl FBIOPAN_DISPLAY to show buffer 2 on screen.

[qiang_li-mpu_se]

The ov10635 and max9286 driver captured frame format is YUYV, that means data in buffer is YUYV, if your application handles it other format, such as RGB565, you can get such wrong color issue.

Another thing should be paied attention to is that the MAX96705 can swap the data line order, if you set it to wrong, the color is wrong.

[yafengge126]

Qiang_FSL‌

谢谢回复！

>The ov10635 and max9286 driver captured frame format is YUYV, that means data in buffer is YUYV, if your application handles it other format, such as RGB565, you can >get such wrong color issue

上面图片是使用7yuv查看的结果，选择的格式是YUYV；图像颜色不对。所以不是显示格式不匹配的问题。

>Another thing should be paied attention to is that the MAX96705 can swap the data line order, if you set it to wrong, the color is wrong.

请问您知道驱动那里可以swap dataline order吗？谢谢！

[qiang_li-mpu_se]

MAX96705 data line swap related code are in regiter 0x20~0x29; 0x30~0x39, you can find the discussion from old items here.

[765077233]

Hi Qiang Li,

Sorry, maybe I didn't tell you my question clearly:

1.When I don't use -g2d，the display image is ok，no screen tearing. And the demo code really use FBIOPAN_DISPLAY to change display buffer ;

2.But when I use -g2d，screen tearing occurs. And the demo code use g2d_blit to display. So my question is how to resolve this problem?

Thanks!