For iMX6DQ, there are two IPUs, so they can support up to 4 cameras at the same time. But the default BSP can only support up to two cameras at the same time. The attached patch can make the BSP support up to 4 cameras based on 3.10.53 GA 1.1.0 BSP.   The 4 cameras can be: - 1xCSI, 3xMIPI - 2xCSI, 2xMIPI - 4xMIPI   For 4xMIPI case, the four cameras should be combined on the single MIPI CSI2 interface, and each camera data should be transfered on a mipi virtual channel.   In this patch, we given the example driver for Maxim MAX9286, it was verified working on iMX6DQ SabreAuto board. The input to MAX9286 is four 720P30 cameras. The verified camera boards:     (1) Onsemi AR0140+AP0101+MAX9271 boards.     (2) OmniVision OV10635+MAX9271 boards.   The MIPI CSI2 CVBS camera surround view solution can be found at: iMX6DQ ISL79985/79987 MIPI CSI2 CVBS camera surround view solution for Linux BSP The MIPI CSI2 CVBS HD camera surround view solution can be found at: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP   The kernel patches: 0001-IPU-update-IPU-capture-driver-to-support-up-to-four-.patch      Updated IPU common code to support up to four cameras.   0002-Add-Max9286-support-on-SabreAuto-board-which-can-sup.patch      MAX9286 driver, it includes MAX9271, AP0101 and AR0140 drivers.   0003-Remove-the-page-size-align-requirement-for-v4l2-capt.patch      With this patch, the mxc_v4l2_tvin test application can use overlay framebuffer as V4l2 capture buffer directly.   0004-Max9286-skip-AP0101-camera-re-initialization.patch      If the camera board's power had been kept after initialized, this patch will bypass the re-initialization to reduce the start up time.   0005-Max9286-set-I2C-speed-to-400Kbps.patch     Set I2C to 400Kbps to reduce the AP0101+AR0140 initialization time.   0006-Max9286-add-retry-for-MAX9271-I2C-access.patch     Added retry for MAX9271 I2C access.   0007-Max9286-Add-support-for-OV10635-camera.patch     Updated code for OV10635 camera.   0008-Max9286-support-auto-detect-camera-number.patch     Make the Max9286 driver can detect the camera number automatically.     How to builld the kernel with MAX9286 support:       make imx_v7_defconfig       make menuconfig (In this command, you should select the MAX9286 driver:             Device Drivers  --->                   <*> Multimedia support  --->                         [*]   V4L platform devices  --->                               <*>   MXC Video For Linux Video Capture                                       MXC Camera/V4L2 PRP Features support  --->                                           <*>Maxim max9286 GMSL Deserializer Input support                                               Select Camera Sensor (OmniVision OV10635 camera sensor)  // Or (Onsemi AP0101 and AR0140 camera sensor)                                           <*>mxc VADC support                                           <*>Select Overlay Rounting (Queue ipu device for overlay library)                                           <*>Pre-processor Encoder library                                           <*>IPU CSI Encoder library)       make zImage       make dtbs   The built out image file:       arch/arm/boot/dts/imx6q-sabreauto.dtb       arch/arm/boot/zImage   "mxc_v4l2_tvin_max9286.tar.gz" is the test application, test command to capture the four cameras and render on 1080P HDMI display: /mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d & /mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &   Some hardware check point on AR0140+AP0101+MAX9271 camera board (Please get MAX9286 and OV10635 schematics from Maxim): 1. In this patch, MAX9286's I2C address is 0x4D, so ADD0 and ADD1 should be connected to high. AP0101's I2C address is 0xBA, so SADDR should be connected to high.   2. AP0101's DOUT0~DOUT7 should be connected to MAX9271's DIN7~DIN0, the order should be switched, MSB connected to LSB.   3. MAX9271's GPO pin should be connected to AP0101's FRAME_SYNC pin. The pull down resistance on FRAME_SYNC pin should not be 0 ohm.   Some known limitation: 1. AP0101's VSYNC invalid time, last video line's HSYNC to VSYNC porch's max value is 255 pixel clocks, it is not enough for MAX9286 to generate the Frame End MIPI packets for each camera. So in order to let iMX6DQ to capture 1280x720 video for each camera, we had let AP0101 output 1280*724 frame size, and iMX6 will only capture 720 lines, the remained video data and Frame End will be ignored. This solution will not impact the function, but there will be "Error matching Frame Start with Frame End for Virtual Channel x" error reported from iMX6 MIPI_CSI_ERR1 register. Maxim suggested to use MAX96705 to relace the MAX9271, it can delay the VSYNC invalid time, then the MIPI error will be fixed.     2015-11-17 update: Updated for OV10635 camera support. File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-11-17.zip   2015-12-04 update: File: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2015-12-04.zip Added patch 0009-Max9286-updated-PCLK-edge-setting-for-OV10635.patch to correct the OV10635 PCLK edge setting     2016-03-07 update: File L3.14.38_GA_MAX9286_Surroundview_Patch_2016-03-07.zip Added kernel patch for L3.14.38 GA 1.1.0 BSP.   2016-07-26 update: Files: L3.10.53_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.38_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip; L3.14.52_GA1.1.0_MAX9286_Surroundview_Patch_2016-07-26.zip. Added gstreamer support. Added MAX96705 support. Added patch for L3.14.52_GA1.1.0.   2017-12-11 update: Added CVBS surround view link: iMX6DQ TP2854 MIPI CSI2 720P CVBS camera surround view solution for Linux BSP     2021-04-26 update: Some customer reported, when system loading is heavy, sometimes, some camera will flicker left and right. It is caused by SFMC FIFO data lost. The original patch used IDMAC 0 and IDMAC 1 for two cameras on one IPU, this is not the best setting.  IDMAC 1 is fixed to use 1/4 SMFC FIFO and it will cause IDMAC 0 to use 1/4 SMFC FIFO too. And another 1/2 of SMFC FIFO can't be used in this case. Some code update to improve it: For each IPU, please use IDMAC 0 and IDMAC 2 to capture the two cameras. This needs change the hard coding in "drivers\media\platform\mxc\capture\ipu_csi_enc.c", "CSI_MEM1" and "IPU_IRQ_CSI1_OUT_EOF" should be changed to "CSI_MEM2" and "IPU_IRQ_CSI2_OUT_EOF". In this case, all SMFC FIFO can be used. And in "ipu_common.c", function ipu_probe(), the followed code should be changed to make IDMAC2 use high priority too. /* Set sync refresh channels and CSI->mem channel as high priority */ - ipu_idmac_write(ipu, 0x18800003L, IDMAC_CHA_PRI(0)); + ipu_idmac_write(ipu, 0x1880000FL, IDMAC_CHA_PRI(0));

Here are two patches to support BT656 and BT1120 output for i.MX6 ipuv3. With this patch, the i.MX6 can support the CVBS output on TV encoder. It is useful for a TV box. "L3.0.35_1.1.0_GA_bt656_output_patch.zip" is the patch for Freescale L3.0.35_1.1.0_GA_iMX6DQ BSP. "r13.4.1_bt656_output_patch.zip" is the patch for Freescale Android R13.4.1 BSP. 1. Features supported:     1) Support BT656(8 bits) and BT1120 (16 bits)interlaced output on display port.     2) Support both RGB and YUV frame buffer for BT656/BT1120 output.     3) Support PAL and NTSC mode.     4) Support on the fly switch between PAL and NTSC mode.     5) Support CVBS output based on adv7391 TV encoder. 2. Hardware link between iMX6 and adv7391 TV encoder chip.     IPU1_DI0_DISP_CLK connected to adv7391 CLKIN pin.     IPU1_DISP0_DAT_23~DISP0_DAT_16 connected to adv7391 P7~P0 pins.     IPU1_DI0_PIN2 connected to adv7391 HSYNC pin. (option)     IPU1_DI0_PIN4 connected to adv7391 VSYNC pin. (option)   - Android R13.4.1 kernel. 3. How to use -- Copy the two patch files to kernel folder.     $ git apply ./0001-Support-BT656-and-BT1120-output-for-iMX6-ipuv3.patch     $ git apply ./0002-Support-adv739x-TV-encoder-for-BT656-output.patch -- Select them in kernel config and build the new kernel image:                     Device Drivers  --->                       Graphics support  --->                           [*]   MXC BT656 and BT1120 output                           [*]   ADV7390/7391 TV Output Encoder -- Uboot parameters for video mode    Output BT656 NTSC data to display port with UVYV frame buffer mode:       "video=mxcfb0:dev=bt656,BT656-NTSC,if=BT656,fbpix=UYVY16"    Output BT656 NTSC data to display port with RGB565 frame buffer mode:       "video=mxcfb0:dev=bt656,BT656-NTSC,if=BT656,fbpix=RGB565"    Output BT656 PAL data to display port with RGB24 frame buffer mode:       "video=mxcfb0:dev=bt656,BT656-PAL,if=BT656,fbpix=RGB24"    Output CVBS NTSC signal on adv7391 with UYVY frame buffer mode:       "video=mxcfb0:dev=adv739x,BT656-NTSC,if=BT656,fbpix=UYVY16"    Output CVBS PAL signal on adv7391 with RGB565 frame buffer mode:       "video=mxcfb0:dev=adv739x,BT656-PAL,if=BT656,fbpix=RGB565" -- Switch between PAL and NTSC    $ echo D:720x480i-60 > /sys/class/graphics/fb0/mode    $ echo D:720x576i-50 > /sys/class/graphics/fb0/mode 4. Note     1) For 8 bits BT656 interface, the default data pins are "DISP0_DAT_23~DISP0_DAT_16", it can also        be any other continued display data pins, for example if "DISP0_DAT_7~DISP0_DAT_0" are used, the        macro "BT656_IF_DI_MSB" in "kernel_imx/drivers/mxc/ipu3/ipu_disp.c" should be changed from "23"        to "7".     2) For 16 bits BT1120 interface, the default data pins are "DISP0_DAT_23~DISP0_DAT_8", it can also        be any other continued display data pins, the macro "BT656_IF_DI_MSB" should be modified if the        hardware pins are changed.     3) When bt656 interface is the second display for each IPU,1-layer-fb (it can be checked with command        "$ cat /sys/class/graphics/fbx/fsl_disp_propperty"), the frame buffer can only be YUV format. In this        case, the IPU DC channel was used for BT656 display, it has no CSC function, so RGB frame buffer was        not supported. 2013-08-09 updated: The new release package "L3.0.35_1.1.0_GA_bt656_output_patch_2013-08-09.zip" had fixed the BT656 dual display issue on iMX6S/DL. Removed the old release package. 2013-09-04 updated: The new release package "r13.4.1_bt656_output_patch_2013-09-04.zip" had fixed the BT656 dual display issue on iMX6S/DL. For default, the dual display was tested with HDMI + CVBS, HDMI is the main display and adv739x CVBS output is the second display. For iMX6DQ which has two IPUs, please assign dual display to two IPUs, for example adv739x is on IPU1 DI0, it is fixed, because hardware pins used for it is fixed. Then we can assign HDMI or LVDS to another IPU (IPU2). For iMX6S/DL which has only one IPU, since adv739x had used IPU1 DI0, another display should be IPU1 DI1. 2013-09-30 updated: Added patch for L3.0.35_4.1.0_GA BSP, the file is "L3.0.35_4.1.0_GA_bt656_output_patch_2013-09-30.zip". 2014-07-21 updated: Added patch for L3.10.17_1.0.0_GA BSP, the file is "L3.10.17_1.0.0_GA_bt656_output_patch_2014-07-21.zip". 2015-01-26 updated: Updated the IPU microcode for 1080i50 and 1080i60 BT1120 output, the parameters "N" for command BMA is a 8 bits parameters, so its max value is 255, but for 1080i50 and 1080i60 output, it needs more blank data in each line, the "N" will be bigger than 255, the updated IPU microcode can fix this limitation. The updated file is "IPU_Microcode_Update_for_BT1120_1080i_20150126.zip". You can update the macro "DC_MCODE_BT656_xxx"  and function _ipu_dc_setup_bt656_interlaced() to the old patch if you used BT1120 mode to support 1080i display. The verified 1080i display mode is: {    /* 1080I60 Interlaced output */   "BT1120-1080I60", 30, 1920, 1080, 13468,   20, 3,   20, 2,   280, 1,   FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT,   FB_VMODE_INTERLACED,   FB_MODE_IS_DETAILED,}, {   /* 1080I50 Interlaced output */   "BT1120-1080I50", 25, 1920, 1080, 13468,   20, 3,   20, 2,   720, 1,   FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT,   FB_VMODE_INTERLACED,   FB_MODE_IS_DETAILED,}, 2016-01-28 updated: Updated IPU microcode to align with BT656.4 specification for NTSC output. For other BSP version with NTSC format support, please reference to ipu_disp_update.c for the final microcode. File "L3.0.35_4.1.0_GA_bt656_output_patch_20160128.zip"., Details, please reference to the readme.txt file in the package. 2016-06-24 update: Added BT656 and BT1120 progressive mode support. File "L3.0.35_4.1.0_GA_bt656_output_patch_20160624.zip". Details, please reference to the readme.txt file in the package. The patch for 3.14.52 GA1.1.0 BSP will be released in next week. 2016-06-27 update: Add BT656 and BT1120 display patch for 3.14.52 BSP. File "L3.14.52_1.1.0_GA_bt656_output_patch_2016-06-27.zip", details, please reference to the readme.txt in the package. 2017-03-10 update: Fixed a hard coding DC macro issue for progressive mode. Added patch "0008-Fixed-a-hard-coding-DC-macro-issue-for-progressive-m.patch" in L3.0.35_4.1.0_GA_bt656_output_patch_2017-03-10.zip. The code in patch "L3.14.52_1.1.0_GA_bt656_output_patch_2016-06-27" is correct.

This patch made the display no interrupt from uboot to kernel to Android. The IPU and related hardware display interface will only be initialized once in Uboot, the kernel code will skip the IPU initialization.   1. Description     1) Support HDMI, LVDS and LCD output in UBoot.     2) Support UBoot logo keep from uboot to kernel to Android.     3) For HDMI, both 720P and 1080P mode were supported.     4) For LVDS, 1024x768 and 1080P dual channel panels were supported.     5) The logo file is a 32 bpp bmp file. 2. File List -- kernel_imx\0001-Keep-uboot-logo-for-Android-boot-supports-HDMI-LCD-a.patch -- kernel_imx\0002-Bug-fix-for-uboot-logo-keep-patch.patch    Kernel patch to support the logo keep feature. -- uboot-imx\0001-Enable-uboot-logo-for-HDMI-LCD-and-LVDS.patch    Uboot patch to support the logo display. -- logo.bmp    Example 32bpp logo file. -- readme.txt    this file, please refer to it before use the patches 3. Requirement - iMX6 SabreSD board. - Android JB4.2.2_1.1.0-GA UBoot and kernel. 4. How to use -- Copy the two patch files to Android kernel_imx and uboot-imx folder and apply them.     $ cd ~/myandroid/kernel_imx/     $ git apply ./0001-Keep-uboot-logo-for-Android-boot-supports-HDMI-LCD-a.patch     $ cd ~/myandroid/bootable/bootloader/uboot-imx/     $ git apply ./0001-Enable-uboot-logo-for-HDMI-LCD-and-LVDS.patch     $ git apply ./0002-Bug-fix-for-uboot-logo-keep-patch.patch   -- Build the new uboot image:     $ cd ~/myandroid/bootable/bootloader/uboot-imx     $ export CROSS_COMPILE=~/myandroid/prebuilt/gcc/linux-x86/arm/arm-eabi-4.6/bin/arm-eabi-     $ export ARCH=arm     $ make mx6q_sabresd_android_config     $ make   -- Before build new UBoot image, the display type can be selected from file uboot-imx\include\configs\mx6q_sabresd.h // Select one of the output mode #define IPU_OUTPUT_MODE_HDMI //#define IPU_OUTPUT_MODE_LVDS //#define IPU_OUTPUT_MODE_LCD   -- Build the new kernel image:     $ cd ~/myandroid/kernel_imx     $ export CROSS_COMPILE=~/myandroid/prebuilt/gcc/linux-x86/arm/arm-eabi-4.6/bin/arm-eabi-     $ export ARCH=arm     $ make imx6_android_defconfig     $ make uImage   -- Before "make uImage", make menuconfig can be used to select the display type.                 System Type  --->                    Freescale MXC Implementations  --->                       MX6 clk setting for smooth UI transtion from bootloader to kernel  --->                           Select Display Interface                              ( )  Smooth UI transtion on LCD, IPU1, DI0                              ( )  Smooth UI transtion on LVDS, IPU1, DI1                              (X)  Smooth UI transtion on HDMI, IPU2, DI0   -- Uboot parameters for video mode    1080P HDMI:       "video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24,bpp=32 fb0base=0x27b00000 fbmem=28M hdmi_audio_clk=148500000"      720P HDMI:       "video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24,bpp=32 fb0base=0x27b00000 fbmem=28M hdmi_audio_clk=74250000"      1024x768 LVDS:       "video=mxcfb0:dev=ldb,LDB-XGA,if=RGB666,bpp=32 fb0base=0x27b00000 fbmem=28M"      800x480 LCD:       "video=mxcfb0:dev=lcd,CLAA-WVGA,if=RGB565,bpp=32 fb0base=0x27b00000 fbmem=28M" -- dd the logo.bmp to SD card address 0x100000 and skip the 54 bytes bmp file header.    sudo dd if=logo.bmp of=/dev/sdc bs=1 seek=1048576 skip=54 5. Note     1) The logo.bmp file should be 32bpp or 16bpp, and it should be synced with video mode parameters "bpp=xx",          and uboot config file mx6q_sabresd.h (#define DISPLAY_BPP  xx).       2) The IPU number and DI number are hard coded in kernel file "board-mx6q_sabresd.c". static struct fsl_mxc_hdmi_core_platform_data hdmi_core_data = {   .ipu_id = 1,   .disp_id = 0, }; static struct fsl_mxc_lcd_platform_data lcdif_data = {   .ipu_id = 0,   .disp_id = 0,   .default_ifmt = IPU_PIX_FMT_RGB565, }; static struct fsl_mxc_ldb_platform_data ldb_data = {   .ipu_id = 0,   .disp_id = 1,   .ext_ref = 1,   .mode = LDB_SEP1,   .sec_ipu_id = 0,   .sec_disp_id = 0, };       3) The IPU number and DI number are defined by Macro in Uboot file "include\configs\mx6q_sabresd.h" #define IPU_NUM   2  // 1 for IPU1, 2 for IPU2. #define DI_NUM   0  // 0 for DI0, 1 for DI1.       4) The display type used in uboot and kernel must be same, same type, same IPU number, same DI port and        same resolution.     [2015-06-29 Update]: JB4.2.2_1.1.0_uboot_logo_keep_patch_2015-06-29.zip Fix some LVDS issues for iMX6DL. Also given an example for LVDS0 with DI0. New Uboot patches:      0002-Updated-lvds-clock-source-to-pll2_pfd0.-Same-as-kern.patch      0003-Add-support-for-iMX6DL.patch   New kernel patches      0003-Skip-lvds-re-initialization-for-logo-keep.patch      0004-Add-examlpe-for-LVDS0-logo-keep.patch     [2015-08-07 Update]: JB4.2.2_1.1.0_uboot_logo_keep_patch_2015-08-07.zip Added the new Uboot patch 0004-Correct-the-sequence-to-set-LDB-clock.patch It can correct the LVDS clock set sequence whch is a known issue that caused no LVDS display sometimes.   [2015-09-18 Update]: JB4.3_1.1.1_uboot_logo_keep_patch_2015-09-18.zip Added the patch for Android JB4.3_GA1.1.1 release. Updated clock usecount, after blank the display, the related clock can be gated off correctly. Support LVDS clock from PLL5.   [2015-12-21 Update]: Added 3.10.53_GA1.1.0 patch: L3.10.53_GA1.1.0_uboot_logo_keep_patch_2015-12-21.zip. Verified on iMX6DL/Q SabreSD board. It supports LCD and LVDS panels, HDMI patch will be released later.   [2016-01-04 Update]: Added 3.10.53_GA1.1.0 patch: L3.10.53_GA1.1.0_uboot_logo_keep_patch_2016-01-04.zip. Added HDMI display support. Now it supports LCD, LVDS and HDMI displays. Fixed the video playback issue for boot up.   [2016-05-18 Update]: 0001-Fix-the-split-mode-LVDS-panel-no-TX3-signal-issue.patch An issue was founded, when dual channel 4 lanes LVDS panel was used, in uboot there will be no LVDS TX3 signa on one LVDS port, the attach "0001-Fix-the-split-mode-LVDS-panel-no-TX3-signal-issue.patch" was used to fix this issue, it is based on JB4.3_1.1.1_uboot_logo_keep_patch_2015-09-18.zip, for other BSP, please port it manually.   [2016-08-29 Update]: 0001-After-reset-IPU-in-SRC-Control-Register-wait-for-res.patch On some iMX6 chip, after reset the IPU in SRC Control Register, enable IPU at once will cause system hang up, to avoid such issue, software needs wait for IPU reset done by polling the SRC register. The attach "0001-After-reset-IPU-in-SRC-Control-Register-wait-for-res.patch" was used to fix this issue, it is based on JB4.3_1.1.1_uboot_logo_keep_patch_2015-09-18.zip + "0001-Fix-the-split-mode-LVDS-panel-no-TX3-signal-issue.patch", for other BSP, please port it manually.   [2017-01-06 Update] Added patch for L4.1.15_GA1.2.0 BSP and Android M6.0.1_GA2.1.0 BSP. Files: L4.1.15_GA1.2.0_uboot_logo_keep_patch_2017-01-06.zip; M6.0.1_2.1.0_uboot_logo_keep_patch_2017-01-06.zip

For iMX6DQ, there are two IPUs, so they can support up to 4 cameras at the same time. But the default BSP can only support up to two cameras at the same time.     The attached patch can make the BSP support up to 4 cameras based on 3.10.53 GA 1.1.0 BSP.   The 4 cameras can be: - 1xCSI, 3xMIPI - 2xCSI, 2xMIPI - 4xMIPI   For 4xMIPI case, the four cameras should be combined on the single MIPI CSI2 interface, and each camera data should be transfered on a mipi virtual channel.   In this patch, we given the example driver for Intersil ISL79985. The input to ISL79985 is four CVBS camera. There are four patches: 0001-IPU-update-IPU-capture-driver-to-support-up-to-four-.patch      Updated IPU common code to support up to four cameras.   0002-Add-Intersil-ISL79985-MIPI-Video-Decoder-Driver-for-.patch      ISL79985 driver, which can support both 1 lanes and 2 lanes mode.   0003-Remove-the-page-size-align-requirement-for-v4l2-capt.patch      With this patch, the mxc_v4l2_tvin test application can use overlay framebuffer as V4l2 capture buffer directly.   0004-IPU-CSI-Drop-1-2-frame-on-MIPI-interface-for-interla.patch      This patch is option, it will drop one field data, so for each camera, the input will be 720*240 30 FPS.   For 720P HD solution, it is based on Maxim MAX9286: iMX6DQ MAX9286 MIPI CSI2 720P camera surround view solution for Linux BSP   How to builld the kernel with ISL79985 support:       make imx_v7_defconfig       make menuconfig (In this command, you should select the ISL79985 driver:             Device Drivers  --->                   <*> Multimedia support  --->                         [*]   V4L platform devices  --->                               <*>   MXC Video For Linux Video Capture                                       MXC Camera/V4L2 PRP Features support  --->                                           <*>Intersil ISL79985 Video Decoder support                                           <*>mxc VADC support                                           <*>Select Overlay Rounting (Queue ipu device for overlay library)                                           <*>Pre-processor Encoder library                                           <*>IPU CSI Encoder library)       make zImage       make dtbs   The built out image file:       arch/arm/boot/dts/imx6q-sabresd.dtb       arch/arm/boot/zImage   "mxc_v4l2_tvin.zip" is the test application, test command to capture the four cameras and render on 1080P HDMI display: /mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d & /mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &   2015-10-10 Update: Updated the test application "mxc_v4l2_tvin_isl79985.tar.gz" to fix the Yocto build errors. Updated ISL79985 register setting "page5, isl79985_write_reg(0x07, 0x46)" in patch "0002-Add-Intersil-ISL79985-MIPI-Video-Decoder-Driver-for-.patch", which can fix the green line issue.   2016-01-25 Update: Added de-interlace support, L3.10.53_ISL79985_Surroundview_Patch_20160125.tar.gz New test capplication for de-interlance: mxc_v4l2_tvin_isl79985_vdi_20160125.tar.gz New test commands: /mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d -m & /mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d -m & /mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d -m & /mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d -m &   Note:  with the 0005-Add-interlaced-mode-capture-for-ISL79985.patch, the V4l2 capture driver will return 720x480 video size, but only odd lines have the video data, they are filled in line skip line mode.     2016-11-21 Update: Added ISL79987 support, L3.10.53_ISL7998x_Surroundview_Patch_20161121.zip New test capplication for de-interlance support: mxc_v4l2_tvin_isl7998x.tar.gz   Test commands (without de-interlace): /mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -g2d & /mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -g2d &   Test commands (with de-interlace, for ISL79987 only): /mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 960 -oh 540 -d 1 -x 0 -m 1 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 0 -ow 960 -oh 540 -d 1 -x 1 -m 1 -g2d & /mxc_v4l2_tvin.out -ol 0 -ot 540 -ow 960 -oh 540 -d 1 -x 2 -m 1 -g2d & /mxc_v4l2_tvin.out -ol 960 -ot 540 -ow 960 -oh 540 -d 1 -x 3 -m 1 -g2d &     Now the same patch can support both ISL79985 and ISL79987, with NTSC CVBS camera, for ISL79985, it captures 60fps 720*240; for ISL79987, it captures 30fps 720*480.   2016-11-22 Update: Added patch for L4.1.15 BSP, it supports both ISL79985 and ISL79987, L4.1.15_ISL7998x_Surroundview_Patch_20161122.zip Test capplication mxc_v4l2_tvin_isl7998x.tar.gz is re-used.

Abstract This is a small tutorial about running a simple OpenCL application in an i.MX6Q. It covers a very small introduction to OpenCL, the explanation of the code and how to compile and run it.   Requirements   Any i.MX6Q board. Linux BSP with the gpu-viv-bin-mx6q package (for instructions on how to build the BSP, check the BSP Users Guide)   OpenCL overview   OpenCL allows any program to use the GPGPU features of the GC2000 (General-Purpose Computing on Graphics Processing Units) that means to use the i.MX6Q GPU processing power in any program.   OpenCL uses kernels which are functions that can be executed in the GPU. These functions must be written in a C99 like code. In our current GPU there is no scheduling so each kernel will execute in a FIFO fashion. iMx6Q GPU is OpenCL 1.1 EP conformant. The Code   The example provided here performs a simple addition of arrays in the GPU. The header needed to use openCL is cl.h and is under /usr/include/CL in your BSP rootfs when you install the gpu-viv-bin-mx6q package. The header is typically included like this: #include <CL/cl.h> The libraries needed to link the program are libGAL.so and libOpenCL.so those are under /usr/lib in your BSP rootfs.   For details on the OpenCL API check the khronos page: http://www.khronos.org/opencl/ Our kernel source is as follows: __kernel void VectorAdd(__global int* c, __global int* a,__global int* b) {      // Index of the elements to add      unsigned int n = get_global_id(0);      // Sum the nth element of vectors a and b and store in c      c[n] = a[n] + b[n]; } The kernel is declared with the signature     __kernel void VectorAdd(__global int* c, __global int* a,__global int* b).   This takes vectors a and b as arguments adds them and stores the result in the vector c. It looks like a normal C99 method except for the keywords kernel and global. kernel tells the compiler this function is a kernel, global tells the compiler this attributes are of global address space. get_global_id built-in function   This function will tell us to which index of the vector this kernel corresponds to. And in the last line the vectors are added. Below is the full source code commented. //************************************************************ // Demo OpenCL application to compute a simple vector addition // computation between 2 arrays on the GPU // ************************************************************ #include <stdio.h> #include <stdlib.h> #include <CL/cl.h> // // OpenCL source code const char* OpenCLSource[] = { "__kernel void VectorAdd(__global int* c, __global int* a,__global int* b)", "{", " // Index of the elements to add \n", " unsigned int n = get_global_id(0);", " // Sum the nth element of vectors a and b and store in c \n", " c[n] = a[n] + b[n];", "}" }; // Some interesting data for the vectors int InitialData1[20] = {37,50,54,50,56,0,43,43,74,71,32,36,16,43,56,100,50,25,15,17}; int InitialData2[20] = {35,51,54,58,55,32,36,69,27,39,35,40,16,44,55,14,58,75,18,15}; // Number of elements in the vectors to be added #define SIZE 100 // Main function // ************************************************************ int main(int argc, char **argv) {      // Two integer source vectors in Host memory      int HostVector1[SIZE], HostVector2[SIZE];      //Output Vector      int HostOutputVector[SIZE];      // Initialize with some interesting repeating data      for(int c = 0; c < SIZE; c++)      {           HostVector1[c] = InitialData1[c%20];           HostVector2[c] = InitialData2[c%20];           HostOutputVector[c] = 0;      }      //Get an OpenCL platform      cl_platform_id cpPlatform;      clGetPlatformIDs(1, &cpPlatform, NULL);      // Get a GPU device      cl_device_id cdDevice;      clGetDeviceIDs(cpPlatform, CL_DEVICE_TYPE_GPU, 1, &cdDevice, NULL);      char cBuffer[1024];      clGetDeviceInfo(cdDevice, CL_DEVICE_NAME, sizeof(cBuffer), &cBuffer, NULL);      printf("CL_DEVICE_NAME: %s\n", cBuffer);      clGetDeviceInfo(cdDevice, CL_DRIVER_VERSION, sizeof(cBuffer), &cBuffer, NULL);      printf("CL_DRIVER_VERSION: %s\n\n", cBuffer);      // Create a context to run OpenCL enabled GPU      cl_context GPUContext = clCreateContextFromType(0, CL_DEVICE_TYPE_GPU, NULL, NULL, NULL);      // Create a command-queue on the GPU device      cl_command_queue cqCommandQueue = clCreateCommandQueue(GPUContext, cdDevice, 0, NULL);      // Allocate GPU memory for source vectors AND initialize from CPU memory      cl_mem GPUVector1 = clCreateBuffer(GPUContext, CL_MEM_READ_ONLY |      CL_MEM_COPY_HOST_PTR, sizeof(int) * SIZE, HostVector1, NULL);      cl_mem GPUVector2 = clCreateBuffer(GPUContext, CL_MEM_READ_ONLY |      CL_MEM_COPY_HOST_PTR, sizeof(int) * SIZE, HostVector2, NULL);      // Allocate output memory on GPU      cl_mem GPUOutputVector = clCreateBuffer(GPUContext, CL_MEM_WRITE_ONLY,      sizeof(int) * SIZE, NULL, NULL);      // Create OpenCL program with source code      cl_program OpenCLProgram = clCreateProgramWithSource(GPUContext, 7, OpenCLSource, NULL, NULL);      // Build the program (OpenCL JIT compilation)      clBuildProgram(OpenCLProgram, 0, NULL, NULL, NULL, NULL);      // Create a handle to the compiled OpenCL function (Kernel)      cl_kernel OpenCLVectorAdd = clCreateKernel(OpenCLProgram, "VectorAdd", NULL);      // In the next step we associate the GPU memory with the Kernel arguments      clSetKernelArg(OpenCLVectorAdd, 0, sizeof(cl_mem), (void*)&GPUOutputVector);      clSetKernelArg(OpenCLVectorAdd, 1, sizeof(cl_mem), (void*)&GPUVector1);      clSetKernelArg(OpenCLVectorAdd, 2, sizeof(cl_mem), (void*)&GPUVector2);      // Launch the Kernel on the GPU      // This kernel only uses global data      size_t WorkSize[1] = {SIZE}; // one dimensional Range      clEnqueueNDRangeKernel(cqCommandQueue, OpenCLVectorAdd, 1, NULL,      WorkSize, NULL, 0, NULL, NULL);      // Copy the output in GPU memory back to CPU memory      clEnqueueReadBuffer(cqCommandQueue, GPUOutputVector, CL_TRUE, 0,      SIZE * sizeof(int), HostOutputVector, 0, NULL, NULL);      // Cleanup      clReleaseKernel(OpenCLVectorAdd);      clReleaseProgram(OpenCLProgram);      clReleaseCommandQueue(cqCommandQueue);      clReleaseContext(GPUContext);      clReleaseMemObject(GPUVector1);      clReleaseMemObject(GPUVector2);      clReleaseMemObject(GPUOutputVector);      for( int i =0 ; i < SIZE; i++)           printf("[%d + %d = %d]\n",HostVector1[i], HostVector2[i], HostOutputVector[i]);      return 0; } How to compile in Host   Get to your ltib folder and run $./ltib m shell This way you will be using the cross compiler ltib uses and the default include and lib directories will be the ones in your bsp. Then run LTIB> gcc cl_sample.c -lGAL -lOpenCL -o cl_sample. How to run in the i.MX6Q   Insert the GPU module root@freescale/home/user $ modprobe galcore Copy the compiled CL program and then run root@freescale /home/user$ ./cl_sample References   [1] ttp://www.khronos.org/opencl/ Original Attachment has been moved to: libOpenCL.so.zip Original Attachment has been moved to: libCLC_Android.so.zip Original Attachment has been moved to: libOpenCL_Android.so.zip Original Attachment has been moved to: libCLC.so.zip

Multiple-Overlay (or Multi-Overlay) means several video playbacks on a single screen. In case multiple screens are needed, check the dual-display case GStreamer i.MX6 Multi-Display $ export VSALPHA=1 $ SAMPLE1=sample1.avi; SAMPLE2=sample2.avi; SAMPLE3=sample3.avi; SAMPLE4=sample4.avi; $ WIDTH=320; HEIGHT=240; SEP=20 Four displays (2x2) $gst-launch \ playbin2 uri=file://`pwd`/$SAMPLE1 video-sink="mfw_isink axis-top=0 axis-left=0   disp-width=$WIDTH disp-height=$HEIGHT" \ playbin2 uri=file://`pwd`/$SAMPLE2 video-sink="mfw_isink axis-top=0 axis-left=`expr $WIDTH + $SEP` disp-width=$WIDTH disp-height=$HEIGHT" \ playbin2 uri=file://`pwd`/$SAMPLE3 video-sink="mfw_isink axis-top=`expr $HEIGHT + $SEP` axis-left=0   disp-width=$WIDTH disp-height=$HEIGHT" \ playbin2 uri=file://`pwd`/$SAMPLE4 video-sink="mfw_isink axis-top=`expr $HEIGHT + $SEP` axis-left=`expr $WIDTH + $SEP` disp-width=$WIDTH disp-height=$HEIGHT" Basic rotation, (2 x 1, normal and inverted) gst-launch \ playbin2 uri=file://`pwd`/$SAMPLE1 video-sink="mfw_isink axis-top=0 axis-left=0   disp-width=$WIDTH disp-height=$HEIGHT rotation=0" \ playbin2 uri=file://`pwd`/$SAMPLE2 video-sink="mfw_isink axis-top=`expr $HEIGHT + $SEP` axis-left=0 disp-width=$WIDTH disp-height=$HEIGHT rotation=3"

Graphics are a big topic in the Android platform, containing java/jni graphic framework and 2d/3d graphic engines (skia, OpenGL-ES, renderscript). This document describes the general Android graphic stack and UI features on Freescale devices. 1. Android Graphic Stacks All Android 3D apps and games have the following graphic stack: Android system UI and all Apps UI follow 2D graphic stack as below, the hardware render will accelerate Android 2D UI with GPU HW OpenGL-ES 2.0 to improve the whole UI performance. Hardware acceleration can be disabled on i.mx6 in device/fsl/imx6/soc/imx6dq.mk USE_OPENGL_RENDERER := false Then rebuild frameworks/base/core/jni, and replace libandroid_runtime.so Surfaceflinger is responsible of all surface layers composition, and  then generate the framebuffer pixmap for display devices. these graphic surface layers are from 2D/3D apps. Hwcomposer is the alternative module of Surfaceflinger with OpenGL-ES. Hwcomposer is used to combine the specific surface layers supported by specific vendor devices. Freescale i.MX6 devices use GPU 2D to combine most surface layers, and the system power can be reduced with GPU 2D instead of GPU 3D. The typical power saving case is video playback. Hwcomposer with GPU 2D can offload GPU 3D task when running game and benchmarks, it is proved to improve the overall system performance about 20%. 2. Performance measurment Show FPS for Android system performance For NFS boot you can set “debug.sf.showfps” to 1 in init.freescale.rc (“setprop debug.sf.showfps 1”) and then reboot the system. For SD or EMMC boot, you can issue command “setprop debug.sf.showfps 1” in console, then find system_server thread by top and kill it to reset the system. Graphic benchmarks for 3D capability measurement Quadrant Full test benchmark cover CPU, Memory, IO, 2D and 3D GLBenchmark http://www.glbenchmark.com/ NenaMark2 https://market.android.com/details?id=se.nena.nenamark2 An3DBench http://www.androidzoom.com/android_applications/tools/an3dbench_hnog.html AnTutu http://www.antutu.com/software.html 3DMark http://www.futuremark.com/benchmarks/3dmark06/introduction/ Browser benchmarks http://www.webkit.org/perf/sunspider/sunspider.html http://v8.googlecode.com/svn/data/benchmarks/current/run.html http://www.craftymind.com/guimark2/ http://www.craftymind.com/factory/guimark/GUIMark_HTML4.html http://themaninblue.com/writing/perspective/2010/03/22/ 3.  Android UI features Dual display with same content This feature is supported in the default image in Android i.MX 6 release package. In this feature, LVDS panel and HDMI output can be supported simultaneously. It is only enabled when the HDMI TV has been connected with the board. Overscan for TV devices Some TVs may miss display the contents in overscan area. To avoid the contents in overscan area being lost, the common implement is by underscanning with an adjustable black border and letitng the viewer adjust the width of the black border. The downscan operation is done by surfaceflinger when it does surface composition through HW OpenGL ES. There is no performance impact since all the work is done by GPU HW. Overscan can be configured in display setting in visual mode: 32 bits color depth 32bpp UI can be supported by adding “bpp=32” in uboot as below: setenv bootargs ‘… video=mxcdi1fb:RGB666,XGA,bpp=32 …’, also can configure it in display setting. Enable 32bpp frame buffer and application surface buffer will be allocate to RGBA8888 format instead of default RGB565 format, that means more system memory is allocated. After enabling 32bpp, if some applications still don't have better UI quality, check to see if  there is hard code to request RGB565 format surface (should request RGBA8888 format to get better quality). Sample code is attached to test for 32bpp (left is on 16bpp, right is on 32bpp) Display Visual Setting The display setting is the add-on feature in FSL Android release, it is very convenient for end-users to change display property, mostly for the following features: Dual display enablement Display color depth setting(16bpp, 32bpp) Overscan adjustment in horizontal and vertical orientation 4. Issue Diagnosis Application Compatibility Some Android applications may not run correctly on some Android releases. It may cause application compatibility, so check the application in other platforms. For example Neocore and Asphalt 5 can run on Eclair, Froyo, and Gingerbread, but will not correctly run on Honeycomb. GPU Compatibility Some game UIs may not correctly display on our Android release. When encountering this kind of issue, the customer can check whether it is caused by the game using an OpenGL extension which our GPU does not support. They can download another data package (for example not extension data package) to have a check. Others Enlarge GPU memory if you encounter UI abnormally displaying after running an application for a while. Some applications need Wifi connections, so monitor the console log to see whether there are any error reports.

  Test environment   i.MX8MP EVK LVDS0 LVDS-HDMI  bridge(it6263) L5.15.5_1.0.0 Background   Some customers need show logo using LVDS panel. Current BSP doesn't support LVDS driver in Uboot. This patch provides i.MX8MPlus LVDS driver support in Uboot. If you want to connect it to LVDS panel , you need port your lvds panel driver like  simple-panel.c   Update [2022.9.19] Verify on L5.15.32_2.0.0  0001-L5.15.32-Add-i.MX8MP-LVDS-driver-in-uboot 'probe device is failed, ret -2, probe video device failed, ret -19' is caused by below code. It has been merged in attachment. // /* Only handle devices that have a valid ofnode */ // if (dev_has_ofnode(dev) && !(dev->driver->flags & DM_FLAG_IGNORE_DEFAULT_CLKS)) { // /* // * Process 'assigned-{clocks/clock-parents/clock-rates}' // * properties // */ // ret = clk_set_defaults(dev, CLK_DEFAULTS_PRE); // if (ret) // goto fail; // }   [2023.3.14] Verify on L5.15.71 0001-L5.15.71-Add-i.MX8MP-LVDS-support-in-uboot   [2023.9.12] For some panel with low DE, you need uncomment CTRL_INV_DE line and set this bit to 1. #include <linux/string.h> @@ -110,9 +111,8 @@ static void lcdifv3_set_mode(struct lcdifv3_priv *priv, writel(CTRL_INV_HS, (ulong)(priv->reg_base + LCDIFV3_CTRL_SET)); /* SEC MIPI DSI specific */ - writel(CTRL_INV_PXCK, (ulong)(priv->reg_base + LCDIFV3_CTRL_CLR)); - writel(CTRL_INV_DE, (ulong)(priv->reg_base + LCDIFV3_CTRL_CLR)); - + //writel(CTRL_INV_PXCK, (ulong)(priv->reg_base + LCDIFV3_CTRL_CLR)); + //writel(CTRL_INV_DE, (ulong)(priv->reg_base + LCDIFV3_CTRL_CLR)); }       [2024.5.15] If you are uing simple-panel.c, need use below patch to set display timing from panel to lcdif controller. diff --git a/drivers/video/simple_panel.c b/drivers/video/simple_panel.c index f9281d5e83..692c96dcaa 100644 --- a/drivers/video/simple_panel.c +++ b/drivers/video/simple_panel.c @@ -18,12 +18,27 @@ struct simple_panel_priv { struct gpio_desc enable; }; +/* define your panel timing here and + * copy it in simple_panel_get_display_timing */ +static const struct display_timing boe_ev121wxm_n10_1850_timing = { + .pixelclock.typ = 71143000, + .hactive.typ = 1280, + .hfront_porch.typ = 32, + .hback_porch.typ = 80, + .hsync_len.typ = 48, + .vactive.typ = 800, + .vfront_porch.typ = 6, + .vback_porch.typ = 14, + .vsync_len.typ = 3, +}; + @@ -100,10 +121,18 @@ static int simple_panel_probe(struct udevice *dev) return 0; } +static int simple_panel_get_display_timing(struct udevice *dev, + struct display_timing *timings) +{ + memcpy(timings, &boe_ev121wxm_n10_1850_timing, sizeof(*timings)); + + return 0; +} static const struct panel_ops simple_panel_ops = { .enable_backlight = simple_panel_enable_backlight, .set_backlight = simple_panel_set_backlight, + .get_display_timing = simple_panel_get_display_timing, }; static const struct udevice_id simple_panel_ids[] = { @@ -115,6 +144,7 @@ static const struct udevice_id simple_panel_ids[] = { { .compatible = "lg,lb070wv8" }, { .compatible = "sharp,lq123p1jx31" }, { .compatible = "boe,nv101wxmn51" }, + { .compatible = "boe,ev121wxm-n10-1850" }, { } };  

The attached patch enables HDMI overscan for Android JB, and tested by MX6Q SabreSD with Android_4.2.2_1.0.0-ga. The bootargs includes "video=mxcfb0:dev=ldb,bpp=32 video=mxcfb1:dev=hdmi,1920x1080M@60,if=RGB24,bpp=32 video=mxcfb2:off".

This is the prototype solution to enable second display showing different things on JB4.2.2 SabreSD. Make use of Class Presentation provided by android to be embedded into Status bar. When unlock the screen, the Presentation will show on second display. Now, the solution requires one .mp4 video placed in root sdcard. Of course, you may change it to show anything. The attached Files are a layout xml file, a patch and a recorded video. The layout file should be put into android/frameworks/base/packages/SystemUI/res/layout/ folder. The patch should be applied to frameworks/base.git. The recorded video shows the dual display demo as a reference.

It is based on L3.0.35_GA4.1.0 BSP.   In default Linux BSP, there are 3 kinds of de-interlace mode, motion =0,1,2 mode, motion mode 0 and 1 will use three fields for de-interlace, and motion mode 2 wil use one field for de-interlace, so the whole fps is 30. In this mode, for motion mode 0 and 1, field 1,2,3 was used for first VDI output frame of display; and field 3,4,5 was used for second VDI output frame of display; field 5,6,7 was used for third VDI output frame of display. One field data (such as 2,4,6) was used only once, so there is data lost.   After applied these patches, the VDI de-interlace output will be 60fps: for motion mode 0 and 1, field 0,1,2 was used for first VDI output frame of display; and field 1,2,3 was used for second VDI output frame of display; field 2,3,4 was used for third VDI output frame of display. So all field data will be used twice, there is no video data lost, the VDI quality was improved.   Kernel patches: 0001-Add-MEM-to-VDI-to-MEM-support-for-IPU.patch 0002-Add-IPU-IC-memcpy-support.patch 0003-IPU-VDI-support-switch-odd-and-even-field-in-motion-.patch 0004-IPU-VDI-correct-vdi-top-field-setting.patch   mxc_v4l2_tvin_imx6_vdi_60fps.zip: this is the test application sample code.   Test commands, parameter "-vd" means double fps VDI: ./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 720 -oh 480 -m 0 -vd  

Hardware connection: there are two board-to-board connectors on E-INK daughter card IMXEBOOKDC4, while there is only one on i.MX7D Sabre board, as the picture below. This might be a bit confusing to connect the two: Checked with internal, the original design was trying to wire both eLCDIF and EPDC bus out to one daughter card, add the flexibility to have different configurations on one display daughter card(LCD/EPD). On i.MX7D Sabre board, only one connector is available for EPDC bus. Here is how we connect i.MX7D Sabre and IMXEBOOKDC4: Software setup: here we use pre-build L3.14.38_6UL7D_Beta Linux as our boot-image, steps to setup/boot/test EPDC: 1. download and decompress BSP pre-build image package "L3.14.38_beta_images_MX6UL7D.tar.gz", you should be able to find the SD image in it -- "fsl-image-gui-x11-imx7dsabresd.sdcard" 2. program the SD image on your SD card(>800 MBytes) with command(I'm running this in Ubuntu): "dd if=fsl-image-gui-x11-imx7dsabresd.sdcard of=/dev/sdb;sync" 3. insert SD card to the slot(J6) on i.MX7D Sabre board, connect debug-UART and power-on the board 4. modify the u-boot environment variables as below:      a.) setenv fdt_file imx7d-sdb-epdc.dtb           (originally this is "fdt_file=imx7d-sdb.dtb")      b.) setenv mmcargs 'setenv bootargs console=${console},${baudrate} root=${mmcroot} epdc video=mxcepdcfb:E060SCM,bpp=16'           (originally this is "mmcargs=setenv bootargs console=${console},${baudrate} root=${mmcroot}") 5. boot into Linux kernel, run unit-test: "/unit_tests/mxc_epdc_fb_test.out", should be able to have test patterns running on EPD.

Multiple-Display means video playback on multiple screens. In case playback needs to be in a unique screen, the mfw_isink element must be used and some pipelines examples can be found on this link: GStreamer iMX6 Multi-Overlay. Number of Displays Display type Kernel parameters Pipelines # Set these shells variables before running the pipelines alias gl=gst-launch SINK_1="\"mfw_v4lsink device=/dev/video17\"" SINK_2="\"mfw_v4lsink device=/dev/video18\"" SINK_3="\"mfw_v4lsink device=/dev/video20\"" media1=file:///root/media1 media2=file:///root/media2 media3=file:///root/media3 2 hdmi + lvds video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 gl playbin2 uri=$media1 video-sink=$SINK_1 playbin2 uri=$media2 video-sink=$SINK_2 2 lvds + lvds video=mxcfb0:dev=ldb,LDB-XGA,if=RGB666 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 gl playbin2 uri=$media1 video-sink=$SINK_1 playbin2 uri=$media2 video-sink=$SINK_2 2 lcd + lvds video=mxcfb0:dev=lcd,800x480M@55,if=RGB565 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB666 gl playbin2 uri=$media1 video-sink=$SINK_1 playbin2 uri=$media2 video-sink=$SINK_2 3 hdmi + lvds + lvds video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24 video=mxcfb1:dev=ldb,LDB-XGA,if=RGB6 video=mxcfb2:dev=ldb,LDB-XGA,if=RGB666 gl playbin2 uri=$media1 video-sink=$SINK_1 playbin2 uri=$media2 video-sink=$SINK_2 playbin2 uri=$media3 video-sink=$SINK_3

                                                                                         Watch the Freescale i.MX team boot up Android 5.0 Lollipop in i.mx6 application processors—在线播放—优酷网，视频高清在线观看 The Freescale i.MX Android team has booted up Android 5.0 Lollipop in the SABRE platform for i.mx6 series. Google pushed all of the latest source for its Android release to AOSP on Nov. 5, and the Freescale Android Team started their work. With the previous 6 days to boot Android Lollipop up, the Freescale i.MX Android team enabled the basic features like connectivity, audio/video playback, sensors, inputs and display on day 7! You can see the some changes in the demo video at the beginning of the post. The Freescale i.MX Android team has closely followed almost every version of Android since it is released by AOSP and has good experience on it. Below are some snapshots and pictures for the Android Lollipop.

Note: All these gstreamer pipelines have been tested using a i.MX6Q board with a kernel version 3.0.35-2026-geaaf30e. Tools: gst-launch gst-inspect FSL Pipeline Examples: GStreamer i.MX6 Decoding GStreamer i.MX6 Encoding GStreamer Transcoding and Scaling GStreamer i.MX6 Multi-Display GStreamer i.MX6 Multi-Overlay GStreamer i.MX6 Camera Streaming GStreamer RTP Streaming Other plugins: GStreamer ffmpeg GStreamer i.MX6 Image Capture GStreamer i.MX6 Image Display Misc: Testing GStreamer Tracing GStreamer Pipelines GStreamer miscellaneous

It is based on 3.0.35 GA 4.1.0 BSP.   0001-Correct-mipi-camera-virtual-channel-setting-in-ipu_c.patch It is the updated IPU code for MIPI ID and SMFC setting in ipu_capture.c. These setting should not be combined with MIPI virtual channel value, they shoule be fixed with ID 0.   0002-Use-virtual-channel-3-for-ov5640-mipi-camera-on-iMX6.patch The sample code to modify ov5640_mipi camera to use virtual channel 3 on SabreSD board.   The followed command can be used to verify the mipi camera function after booted into Linux: $ gst-launch mfw_v4lsrc capture-mode=1 device=/dev/video1 ! mfw_v4lsink     2014-09-30 update: Added the patch for 3.10.17_GA1.0.0 BSP. "L3.10.17_1.0.0_mipi_camera_virtual_channel_3.zip"  

Overview As more and more communication required between online and offline, the QR code is widely used in the mobile payment, mobile small apps, industry things identification and etc. The i.MX6UL/ULL has the IP of CSI and PXP for camera connection and image CSC/FLIP/ROTATION acceleration. A LCDIF IP is supporting the display, but no 3D IP support. This means this low power and low end AP is very suitable for the industry HMI segment, which does not require a cool 3D graphic display, but a simple and straightforward GUI for interaction. QR code scanner is one of the use cases in the industry segment, which more and more customer are focusing on. The i.MX6UL CPU freq of i.MX6UL is about 500Mhz, and it does not have GPU IP, so a lightweight GUI and window system is required. Here we recommend the QT with wayland backend (without X11), which would make the window system small and faster than traditional X11 UI. Why chose QT is because of it has open source version, rich components, platform independent, good performance for embedded system and strong development staffs like QtCreator for creating application. How to enable the QT development environment, check this: Enable QT developement for i.MX6UL (v2)  Here I made a QR code scanner demo based on QT5.6 + QZXing (QR/Bar code scan engine) running on the i.MX6UL EVK board with a UVC camera (at least 640x480 resolution is required) and 480x272px LCD. Source code is open here (License Apache2.0): https://github.com/muddog/QRScanner  Implementation To do camera preview and capture, you must think on the gstreamer first, which is easy use and has the acceleration pads which implemented by NXP for i.MX6UL. Yes, it's very easy for you to enable the preview in console like: $ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=640,height=320 ! imxvideoconvert_pxp ! video/x-raw,format=RGB16 ! waylandsink It works under the i.MX6UL EVK, with PXP IP to do color space convert from YUY2 -> RGB16 acceleration, also the potential scaling of the image. The CPU loading of this is about 20-30%, but if you use the component of "videoconvert" to replace the "imxvideoconvert_pxp", we do CSC and scale by CPU, then the loading would increase to 50-60%. The "/dev/video1" is the device node for UVC camera, it may different in your environment. So our target is clear, create such pipeline (with PXP acceleration) in the QT application, and use a appsink to get preview images, do simple "sink" to one QWidget by drawing this image on the widget surface for preview (say every 50ms for 20fps). Then in other thread, we fetch the preview buffer in a fixed frequency (like every 0.5s), then feed it into the ZXing engine to decode the strings inside this image. Here are the class created inside the source code: ScannerQWidgetSink It act as a gstreamer sink for preview rendering. Init the pipeline, create a timer with timeout every 50ms. In the timer handler, we use appsink to copy the camera buffer from gstreamer, and tell the ViewfinderWidget to do update (re-draw event). ViewfinderWidget This class inherit from the QWidget, which draw the preview buffer as a QImage onto it's own surface by using QPainter. The QImage is created at the very begining with the image buffer created by the ScannerQWidgetSink. Because QImage itself does not maintain the image buffer, so the buffer must be alive during it's usage. So we keep this buffer during the ScannerQWidgetSink life cycle, copy the appsink buffer from pipeline to it for preview. MainWindow Create main window, which does not have title bar and border. Start any animation for the red line scan bar. Create instance of DecoderThread and ScannerQWidgetSink. Setup and start them. DecoderThread A infinite loop, to wait for a available buffer released by the ScannerQWidgetSink every 0.5s. Copy the buffer data to it's own buffer (imgData) to avoid any change to the buffer by sink when doing decoding. Then feed this copy of buffer into ZXing engine to get decoder result. Then show on the QLabel. Screenshot under wayland (weston) desktop: Customize Camera instance Now I use the UVC camera which pluged in the USB host, which device node is /dev/video1. If you want to use CSI or other device, please change the construction parameters for ScannerQWidgetSink(): sink = new ScannerQWidgetSink(ui->widget, QString("v4l2src device=/dev/video1")); Image resolution captured and review Change the static member value of ScannerQWidgetSink class: uint ScannerQWidgetSink::CAPTURE_HEIGHT = 480; uint ScannerQWidgetSink::CAPTURE_WIDTH = 640; Preview fps and decoding frequency Find the "framerate=20/1" strings in the ScannerQWidgetSink::GstPipelineInit(), change to your fps. You also have to change the renderTimer start timeout value in the ::StartRender(). The decoding frequency is determined by renderCnt, which determine after how many preview frames showed to feed the decoder. Main window size It's fixed size of main window, you have to change the mainwindow.ui. It's easy to do in the QtCreate Designer. FAQ Why not use CSI camera in demo? Honestly, I do not have CSI camera module, it's also DNP when you buying the board on NXP.com. So a widely used UVC camera is preferred, it's also easy for you to scan QR code on your phone, your display panel etc. Why not use QCamera to do preview and capture? The QCamera class in the Qtmultimedia component uses the camerabin2 gstreamer plugin, which create a very long pipeline for different usage of viewfinder, image capture and video encoder. Camerabin2 would eat too much CPU and memory resource, take picture and recording are very very slow. The preview of 30fps would eat about 70-80% CPU loading even I hacked it using imxvideoconvert_pxp instread of software videoconvert. Finally I give up to implement the QRScanner based on QCamera. How to make sure only one instance of QT app is running? We can use QSharedMemory to create a share memory with a unique KEY. When second instance of app is started, it would check if the share memory with this KEY is created or not. If the shm is there, it means there's already one instance running, it has to exit(). But as the QT mentioned, the QSharedMemory can not be destroyed correctly when app crashed, this means we have to handle each terminate signal, and do delete by ourselves: static QSharedMemory *gShm = NULL; static void terminate(int signum) {    if (gShm) {       delete gShm;       gShm = NULL;    }    qDebug() << "Terminate with signal:" << signum;    exit(128 + signum); } int main(int argc, char *argv[]) {    QApplication a(argc, argv);    // Handle any further termination signals to ensure the    // QSharedMemory block is deleted even if the process crashes    signal(SIGHUP, terminate ); // 1    signal(SIGINT, terminate ); // 2    signal(SIGQUIT, terminate ); // 3    signal(SIGILL, terminate ); // 4    signal(SIGABRT, terminate ); // 6    signal(SIGFPE, terminate ); // 8    signal(SIGBUS, terminate ); // 10    signal(SIGSEGV, terminate ); // 11    signal(SIGSYS, terminate ); // 12    signal(SIGPIPE, terminate ); // 13    signal(SIGALRM, terminate ); // 14    signal(SIGTERM, terminate ); // 15    signal(SIGXCPU, terminate ); // 24    signal(SIGXFSZ, terminate ); // 25    gShm = new QSharedMemory("QRScannerNXP");    if (!gShm->create(4, QSharedMemory::ReadWrite)) {       delete gShm;       qDebug() << "Only allow one instance of QRScanner";       exit(0);    } .....

There is no Freescale GStreamer element which does the JPEG decoding, so we must rely on a standard one, like 'jpegdec'. In case your Linux system was built using LTIB, in order to have the jpegdec element included on the gst-plugin-good, follow these steps: On the LTIB menuconfig, make sure the following packages are selected: gstreamer-plugins-good libjpeg libpng Remove the configure parameters '--disbale-libpng' and '--disable-jpeg' on the file './dist/lfs-5.1/gst-plugins-good/gst-plugins-good.spec' Rebuild and flash your board (or SD card) again. Image display VSALPHA=1 gst-launch filesrc location=sample.jpeg ! jpegdec ! imagefreeze ! mfw_isink Important: non 8 pixel aligned width and height is treated as not supported format in isink plugin.

This document describes all the i.MX 8 MIPI-CSI use cases, showing the available cameras and daughter cards supported by the boards, the compatible Device Trees (DTS) files, and how to enable these different camera options on the i.MX 8 boards. Plus, this document describes some Advanced camera use cases too, such as multiples cameras output using imxcompositor_g2d plugin, GStreamer zero-copy pipelines and V4L2 API extra-controls examples.

There is GPU SDK for i.MX6D/Q/DL/S: IMX_GPU_SDK.  This is to share the experience when compiling the example code from the SDK with Linux BSP release: L3.0.35_1.1.0_121218 and  L3.0.35_4.0.0_130424 . Minimal profile is using and have been verified on both i.MX6Q SDP and i.MX6DL SDP. To start: Please make sure “gpu-viv-bin-mx6q” has been selected in the Package list and compiled to your rootfs. After finished the compilation of the rootfs, you should find some newly added libraries for GLES1.0, GLES2.0, OpenVG and EGL in <ltib>/rootfs/usr/lib However, you should find libOpenVG.so is actually copied from libOepnVG_3D.so: vmuser@ubuntu:~/ltib_src/ltib/rootfs/usr/lib$ ls -al libOpen* -rwxr-xr-x 1 root root 115999 2013-06-06 18:31 libOpenCL.so -rwxr-xr-x 1 root root 515174 2013-06-06 18:31 libOpenVG_355.so -rwxr-xr-x 1 root root 272156 2013-06-06 18:31 libOpenVG_3D.so -rwxr-xr-x 1 root root 272156 2013-06-06 18:31 libOpenVG.so So, in this way, i.MX6D/Q will no use libOpenVG_355.so in the build. Also, if you run NFS, the libOpenVG.so will change to symbolic link:           For example, run on i.MX6Q SDP, it will link to /usr/lib/libOpenVG_355.so                          For example, run on i.MX6DL SDP, it will link to /usr/lib/libOpenVG_3D.so                Then, when you compile the OpenVG example code, it is becoming very confusing.  Thus, it needs to pay attention when doing the compilation.  For example, delete the symbolic link and make copy of the corresponding library: For i.MX6D/Q, please do this: $ sudo /bin/rm libOpenVG.so $ sudo cp libOpenVG_355.so libOpenVG.so For i.MX6S/DL, please do this: $ sudo /bin/rm libOpenVG.so $ sudo cp libOpenVG_3D.so libOpenVG.so To compile the sample code in the GPU SDK, you could refer to iMXGraphicsSDK_OpenGLES2.0.pdf or iMXGraphicsSDK_OpenGLES1.1.pdf in ~/gpu_sdk_v1.00.tar/Documentation/Tutorials to set up the cross compilation environment; which is assuming the LTIB and the rootfs is ready. $ export ROOTFS=/home/vmuser/ltib_src/ltib/rootfs $ export CROSS_COMPILE=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi- For OpenVG: $ cd ~/gpu_sdk_v1.00/Samples/OpenVG $ make -f Makefile.fbdev clean $ make -f Makefile.fbdev $ make -f Makefile.fbdev install The executable will then be copied to this directory: ~/gpu_sdk_v1.00/Samples/OpenVG/bin/OpenVG_fbdev For GLES2.0 $ cd ~/gpu_sdk_v1.00/Samples/ GLES2.0 $ make -f Makefile.fbdev clean $ make -f Makefile.fbdev $ make -f Makefile.fbdev install The executable will then be copied to this directory: ~/gpu_sdk_v1.00/Samples/ GLES2.0/bin/GLES20_fbdev For GLES1.1, please modify the Makefile.fbdev to remove the compilation of example codes "18_VertexBufferObjects" and "19_Beizer" that are not exist. Then, $ cd ~/gpu_sdk_v1.00/Samples/ GLES1.1 $ make -f Makefile.fbdev clean $ make -f Makefile.fbdev $ make -f Makefile.fbdev install The executable will then be copied to this directory: ~/gpu_sdk_v1.00/Samples/ GLES1.1/bin/GLES11_fbdev Finally, you could copy the executable to the rootfs and test on i.MX6Q SDP/SDB or i.MX6DL SDP board. NOTE: the newly added makefiles.tgz contains Makefile.x11 hacked from GLES2.0 example code to make OpenVG to compile and run on Ubuntu 11.10 rootfs.