The i.MX8QuadMax SMARC System On Module integrates Dual Cortex A72 + Quad Cortex A53 Cores, Dual GPU systems, 4K H.265 capable VPU dual failover-ready display controller based i.MX8 QuadMax SoC with on SOM Dual 10/100/1000 Mbps Ethernet PHY, USB 3.0 hub and IEEE 802.11a/b/g/n/ac Wi-Fi & Bluetooth 5.0 module.

Sometimes it is helpful/faster to build a i.MX8MM boot binary outside of the Yocto environment. There are instructions on how to accomplish this on different places, this document tries to provide an example for the i.MX8M Mini LPDDR4 EVK, whenever possible pointing how to build for other boards. For the 8MM SoC a boot image is generated by imx-mkimage tool and requires: - u-boot - ARM trusted firmware image - ddr training firmware 1. Download and Build u-boot: mkdir imx-boot-bin cdimx-boot-bin git clone https://source.codeaurora.org/external/imx/uboot-imx.git cd uboot-imx/ git checkout -b imx_v2019.04_4.19.35_1.1.0 origin/imx_v2019.04_4.19.35_1.1.0 (Optional) Here you can "git log -1" to check that the commit matches SRCREV on the recipe. Next, use the BSP SDK script to setup the cross compilation environment, instructions on how to build it are here. source /opt/fsl-imx-wayland/4.19-warrior/environment-setup-aarch64-poky-linux export ARCH=arm Build make clean Supported boards have configuration files on "configs". Using the LPDDR4 EVK here: make imx8mm_evk_defconfig make 2.   Download and build the ARM Trusted Firmware cd .. git clone https://source.codeaurora.org/external/imx/imx-atf.git cd imx-atf/ git checkout -b imx_4.19.35_1.1.0 origin/imx_4.19.35_1.1.0 (Optional) Again, you can "git log -1" to check that the commit matches SRCREV on the recipe. https://source.codeaurora.org/external/imx/meta-fsl-bsp-release/tree/imx/meta-bsp/recipes-bsp/imx-atf/imx-atf_2.0.bb?h=warrior-4.19.35-1.1.0 Build: make PLAT=imx8mm bl31 If you run into this error: aarch64-poky-linux-ld.bfd: unrecognized option '-Wl,-O1' aarch64-poky-linux-ld.bfd: use the --help option for usage information make: *** [Makefile:712: build/imx8mm/release/bl31/bl31.elf] Error 1 try:  unset LDFLAGS make PLAT=imx8mm bl31 3. Download the LPDDR4 training binaries It is on firmware-imx, recipe is here: https://source.codeaurora.org/external/imx/meta-fsl-bsp-release/tree/imx/meta-bsp/recipes-bsp/firmware-imx?h=warrior-4.19.35-1.1.0 cd .. mkdir firmware-imx cd firmware-imx wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/firmware-imx-8.5.bin chmod a+x firmware-imx-8.5.bin ./firmware-imx-8.5.bin 4. Download imx-mkimage and build the boot image cd .. git clone https://source.codeaurora.org/external/imx/imx-mkimage.git cd imx-mkimage/ git checkout -b imx_4.19.35_1.1.0 origin/imx_4.19.35_1.1.0 (Optional) "git log -1" matches SRCREV on: https://source.codeaurora.org/external/imx/meta-fsl-bsp-release/tree/imx/meta-bsp/recipes-bsp/imx-mkimage/imx-mkimage_git.inc?h=warrior-4.19.35-1.1.0 Now, you can check the build targets and required binaries at iMX8M/soc.mak For the flash_evk for the imx8mm we will need binaries: u-boot: u-boot-spl.bin, u-boot-nodtb.bin, fsl-imx8mm-evk.dtb  ARM trusted firmware: bl31.bin LPDDR4 files: lpddr4_pmu_train_1d_imem.bin lpddr4_pmu_train_1d_dmem.bin lpddr4_pmu_train_2d_imem.bin lpddr4_pmu_train_2d_dmem.bin mkimage for mkimage_uboot Copy all these to imx-mkimage/iMX8M/ cp ../uboot-imx/spl/u-boot-spl.bin iMX8M/ cp ../uboot-imx/u-boot-nodtb.bin iMX8M/ cp ../uboot-imx/arch/arm/dts/fsl-imx8mm-evk.dtb iMX8M/ cp ../imx-atf/build/imx8mm/release/bl31.bin iMX8M/ cp ../firmware-imx/firmware-imx-8.5/firmware/ddr/synopsys/lpddr4_pmu_train_* iMX8M/ cp ../uboot-imx/tools/mkimage iMX8M/mkimage_uboot Build: make SOC=iMX8MM flash_evk Output binary is on ./iMX8M/flash.bin 5. Program on the SD Card: sudo dd if=iMX8M/flash.bin of=/dev/<path to your sd> bs=1024 seek=33

This package is a OTA upgrade implementation for smartlocker in i.MX7ULP kernel. The bootaux command for i.MX7ULP can also be applied to other projects. File description: smartlocker OTA upgrade user manual. Modified u-boot. Modification involves: Add bootaux command. To use this command, the M4 image will be read out from boot partition to TCM_L. (Or DDR and then it will be copied to TCM_L in the command) It took 19ms to read M4 image. Change u-boot default env. If M4 image and zImage read failed, recovery M4 image and zImage will be loaded. patch of u-boot changes. u-boot defconfig for bootaux change. sh script, updater.sh. Example for upgrade package. Power shutdown in copying upgrade files may cause file broken. So currently, we use below copy strategy: Copy upgrade file to target directory as tmp file. Delete target file. Rename tmp file to target file.

         This document will describe how to add open JDK to i.MX yocto BSP. It will take two versions of Linux BSP as an example, one is the lower version of L4.1.15-2.0.0, the other is the latest version of L4.19.35-1.1.0. Adding openjdk-8 to L4.1.15-2.0.0(Ubuntu 16.04 LTS platform) Before adding an open JDK, you must download L4.1.15-2.0.0 BSP according to the i.MX_Yocto_Project_User's_Guide.pdf, and ensure that it can pass the compilation normally, that is to say, there is no error in the compilation. In this example, BSP is compiled using the following command. # DISTRO=fsl-imx-wayland MACHINE=imx6sxsabresd source fsl-setup-release.sh -b build-wayland # bitbake fsl-image-qt5          Then follow the steps below to add openjdk to the yocto layer:   Fetching openjdk-8 from Yocto website # cd ~/imx-release-bsp # cd sources # git clone git://git.yoctoproject.org/meta-java # cd meta-java # git checkout -b krogoth origin/krogoth  [Comment]    Yocto’s version is described in i.MX_Yocto_Project_User's_Guide.pdf 2. Modifying related configurations (1) build-wayland/conf/local.conf Add following lines to the file: # Possible provider: cacao-initial-native and jamvm-initial-native PREFERRED_PROVIDER_virtual/java-initial-native = "cacao-initial-native" # Possible provider: cacao-native and jamvm-native PREFERRED_PROVIDER_virtual/java-native = "cacao-native" # Optional since there is only one provider for now PREFERRED_PROVIDER_virtual/javac-native = "ecj-bootstrap-native" IMAGE_INSTALL_append = " openjdk-8" Save it and exit (2)build-wayland/conf/bblayers.conf Add java layer to the file, like below: BBLAYERS = " \   ${BSPDIR}/sources/poky/meta \   ${BSPDIR}/sources/poky/meta-poky \   \   ${BSPDIR}/sources/meta-openembedded/meta-oe \   ${BSPDIR}/sources/meta-openembedded/meta-multimedia \   \   ${BSPDIR}/sources/meta-fsl-arm \   ${BSPDIR}/sources/meta-fsl-arm-extra \   ${BSPDIR}/sources/meta-fsl-demos \   ${BSPDIR}/sources/meta-java \ "…… Save it and exit. 3. Build openjdk-8 # cd ~/imx-release-bsp # source setup-environment build-wayland #bitbake openjdk-8 -c fetchall          Fetch all packages related to openjdk-8. [error handling]          During downloading packages, you may encounter errors like the following. (1)Fetch fastjar-0.98.tar.gz errors          The error is caused by invalid web address, we can download it from another link, see below: http://savannah.c3sl.ufpr.br/fastjar/fastjar-0.98.tar.gz copy the link to firefox in Ubuntu platform, and it will be downloaded into ~/Downloads # cd ~/imx-release-bsp/downloads # cp ~/Downloads/ fastjar-0.98.tar.gz ./ # touch fastjar-0.98.tar.gz.done   (2)Fetch “classpath-0.93.tar.gz” error          Download it from : http://mirror.nbtelecom.com.br/gnu/classpath/classpath-0.93.tar.gz And copy it to ~/imx-release-bsp/downloads, and create a file named classpath-0.93.tar.gz.done in the directory. # cd ~/imx-release-bsp/downloads # cp ~/Downloads/ classpath-0.93.tar.gz ./ # touch classpath-0.93.tar.gz.done (3) 8 files with tar.bz2 (hotspot-Java jvm)          These similar errors are very likely to be encountered.          These errors are caused by the bad network environment. You can download these packages manually. These are Java virtual machine source packages, i.e. hotspot JVM [Solution] # mkdir ~/temp # cd temp # wget http://www.multitech.net/mlinux/sources/56b133772ec1.tar.bz2 # wget http://www.multitech.net/mlinux/sources/ac29c9c1193a.tar.bz2 # wget http://www.multitech.net/mlinux/sources/1f032000ff4b.tar.bz2 # wget http://www.multitech.net/mlinux/sources/81f2d81a48d7.tar.bz2 # wget http://www.multitech.net/mlinux/sources/0549bf2f507d.tar.bz2 # wget http://www.multitech.net/mlinux/sources/0948e61a3722.tar.bz2 # wget http://www.multitech.net/mlinux/sources/48c99b423839.tar.bz2 # wget http://www.multitech.net/mlinux/sources/bf0932d3e0f8.tar.bz2          Then create .tar.bz2.done files for each package via touch command   # touch 56b133772ec1.tar.bz2.done # touch ac29c9c1193a.tar.bz2.done # touch 1f032000ff4b.tar.bz2.done # touch 81f2d81a48d7.tar.bz2.done # touch 0549bf2f507d.tar.bz2.done # touch 0948e61a3722.tar.bz2.done # touch 48c99b423839.tar.bz2.done # touch bf0932d3e0f8.tar.bz2.done          Like below:          Then copy these files to ~/ fsl-release-bsp/downloads/ # bitbake openjdk-8 -c compile          After openjdk compilation, you will be prompted as follows:          At last , install openjdk-8 to images # bitbake fsl-image-qt5          Done: [Additional description]          The above method of adding openjdk-8 is the steps after BSP compilation. Users can also add openjdk-8 before BSP compilation, and then compile it with BSP          According to steps in i.MX_Yocto_Project_User's_Guide.pdf, After running the following two commands, users can modify bblayers.conf and local.conf directly.          For example, steps below have been validated: … … # repo sync # cd ~/fsl-release-bsp # DISTRO=fsl-imx-x11 MACHINE=imx6qsabresd source fsl-setup-release.sh -b build-x11 # gedit ./conf/bblayers.conf          Add the same contents as above. # gedit ./conf/local.conf          Add the same contents as above. # bitbake fsl-image-gui          During compilation, users may encounter some errors, which can be handled by referring to the methods described above Adding openjdk-8 to L4.19.35-1.1.0(Ubuntu 18.04 LTS Platform) In fact, the steps to add openjdk-8 to l4.19.35 are the same as those described above, and the following steps have been verified. Before adding openjdk-8, i.mx8qxp full image has been compiled with 2 commands below, so we only need to add openjdk-8 here. # DISTRO=fsl-imx-xwayland MACHINE=imx8qxpmek source fsl-setup-release.sh -b build-xwayland # bitbake imx-image-full # cd sources # git clone git://git.yoctoproject.org/meta-java # cd meta-java # git checkout -b warrior origin/warrior          Release L4.19.35_1.1.0 is released for Yocto Project 2.7 (Warrior). # cd ~/imx-release-bsp-l4.19.35 # source setup-environment build-xwayland-imx8qxpmek # gedit ./conf/bblayers.conf          Add meta-java to it.          ……            ${BSPDIR}/sources/meta-java \          ……          Save and exit. # gedit ./conf/local.conf          Add these lines to it.          # Possible provider: cacao-initial-native and jamvm-initial-native PREFERRED_PROVIDER_virtual/java-initial-native = "cacao-initial-native" # Possible provider: cacao-native and jamvm-native PREFERRED_PROVIDER_virtual/java-native = "cacao-native" # Optional since there is only one provider for now PREFERRED_PROVIDER_virtual/javac-native = "ecj-bootstrap-native" IMAGE_INSTALL_append = " openjdk-8" Save and exit.   # cd ~/imx-release-bsp-l4.19.35/build-xwayland-imx8qxpmek # bitbake openjdk-8 -c fetch # bitbake openjdk-8 -c compile [Errors] [Solution] # gedit ./ tmp/work/x86_64-linux/openjdk-8-native/172b11-r0/jdk8u-33d274a7dda0/hotspot/make/linux/Makefile Comment the following lines: ----------------------------------------- check_os_version: #ifeq ($(DISABLE_HOTSPOT_OS_VERSION_CHECK)$(EMPTY_IF_NOT_SUPPORTED),) #       $(QUIETLY) >&2 echo "*** This OS is not supported:" `uname -a`; exit 1; #endif -----------------------------------------          Then continue # cd ~/imx-release-bsp-l4.19.35/build-xwayland-imx8qxpmek # bitbake openjdk-8 -c compile [comment]          Probably similar errors will be encountered during compiling other packages, we can use the same way like above to solve it, see bellow, please! Done:          At last, install openjdk-8 to images. # bitbake imx-image-full          Installation is done. NXP TIC Team  Weidong Sun 12/31/2019

The D-PHY PLL (in the red circle in the picture below) is the PLL that drives the MIPI Clock lane. It must be set in accordance with the video to be sent to the display.   Calculating the video bandwidth The video bandwidth is calculated with the following equation: Pixels per second = Horizontal res. x Vertical res. x Frame rate x Bits per pixel Taking as example the 1080p60 OLED display RM67191: Pixels per second = 1920 x 1080 x 60 x 24 Pixels per second = 2985984000 = 2,98Gpixels/sec Pixel clock calculation The Display pixel clock can be obtained on the display driver. In this example for RM67191, the pixel clock is 132Mpixel/sec, see file: panel-raydium-rm67191.c\panel\drm\gpu\drivers - linux-imx - i.MX Linux kernel  Line 530: .pixelclock = { 66000000, 132000000, 132000000 }, Or the number can be obtained with the following equation: pixel clock = (hactive + hfront_porch + hsync_len + hback_porch) x (vactive + vfront_porch + vsync_len + vback_porch) x frame rate pixel clock = (1080 + 20 + 2 +34) × (1920 + 10 + 2 + 4) x 60 pixel clock = 132000000 (rounded up) Bit clock calculation (clock lane) The mipi-dphy bit_clk is the output clock and is calculated on file sec-dsim.c (line 1283): sec-dsim.c\bridge\drm\gpu\drivers - linux-imx - i.MX Linux kernel  Bit clock can be calculated with the following equation: bit_clk = Pixel clock * Bits per pixel / Number of lanes In the case of 1980p60 (Raydium display), It is:   bit_clk = pixel clock * bits per pixel / number of lanes bit_clk = 132000000 * 24 / 4 bit_clk = 792000000 Other important timing parameters like 'p', 'm', 's' are obtained on the table in the following header file: sec_mipi_dphy_ln14lpp.h\imx\drm\gpu\drivers - linux-imx - i.MX Linux kernel 

In case you missed our recent webinar, you can check out the slides and comment below with any questions.

This doc show: on i.MX8QXP MEK board, configure ov5640 sensor(parallel interface) output 5MP(2592x1944) RAW(Bayer) data at 15fps, and Parallel Capture Subsystem and Image Sensor Interface capture RAW RGB data, and i.MX8QXP GPU debayer RAW data then display image. HW: i.MX8QXP MEK board, MEK base board (to place the parallel camera), ov5640 sensor. SW: Linux 4.14.98_2.0.0 BSP, and patches in this doc.   Configure at camera sensor side A Bayer filter is a color filter array (CFA) for arranging RGB color filters on a square grid of photosensors. The filter pattern is 50% green, 25% red and 25% blue, hence is also called BGGR, RGBG ,GRGB, or RGGB. The ov5640 has an image array capable of operating at up to 15 fps in 5 megapixel (2592x1944) resolution. OV5640 support output formats: RAW(Bayer), RGB565/555/444,CCIR656, YUV422/420, YCbCr422, and compression. To make ov5640 output 5MP RAW data at 15fps, check my kernel patch imx8-ov5640-raw-capture-driver-4.14.98_2.0.0.diff which apply on i.MX Linux 4.14.98_2.0.0 BSP kernel code. Parallel interface ov5640, use ov5640_raw_setting[] array of drivers/media/platform/imx8/ov5640_v3.c. This register setting is come from ov5640 software application note and data sheet. Configure at i.MX8QXP side The Parallel Capture Subsystem consists of the Parallel Capture Interface (BT 656) and associated peripherals. It interfaces to the Parallel CSI sensor. This allows for up to 24 RGB data bits in parallel or for RGB components on consecutive clocks (up to 10-bit color depth). The formats supported are RGB, RAW and YUV 422. Below is Parallel Capture Subsystem diagram: For RAW format data, CSI_CTRL_REG of Parallel Capture Subsystem need configured as my patch, otherwise found cannot get correct data.   The multiple input sources (MIPI CSI, Parallel Capture) captures the pixel data and feeds it to the ISI. The ISI is responsible for capturing and pre-processing the pixel data from multiple input sources and storing them into the memory. Below is ISI diagram: For RAW format data, it should be bypass any processing pipeline of ISI, just use ISI to save it to memory.   Capture test code To capture the RAW data and save it to file, check my patch imx8_ov5640_raw_captupre_test_4.14.98_2.0.0_ga.diff which apply on i.MX Linux 4.14.98_2.0.0 BSP unit test code. Note the usage is: ./imx8_cap.out -of -cam 1 -fr 15 -fmt BA81 -ow 2592 -oh 1944 -num 100   Display RAW data The RAW data cannot be displayed directly, debayer process is needed to get complete red, green, blue color for each pixel. The debayer process if run on CPU, will cost much CPU time. To save CPU time, debayer could done by GPU. The method is, captured RAW data upload to GPU as texture , then GPU will do the debayer, then full color of each pixel will be got, then display it. To upload RAW camera data to GPU with zero memory copy, i will use i.MX8QXP GPU extension GL_VIV_direct_texture. It create a texture with direct access support. API glTexDirectVIVMap,  which support mapping a user space memory or a physical address into the texture surface. The API glTexDirectVIVMap need logic and physical address of data buffer, so i will allocate data buffer from g2d lib, it is dma-buffer also get logic/physical address of buffer, then queue it as DMABUF to v4l2 capture driver, after dequeue got RAW camera data, pass it to GPU for debayer. GPU side, I will use OpenGL shader code from "Efficient, High-Quality Bayer Demosaic Filtering on GPUs". Check my patch imx8_debayer-gpusdk-5.3.0.diff which apply on i.MX GPU SDK 5.3.0 code. Note, here i only do is debayer, no extra process.   Known issue One thing is ov5640 output 5MP at 15fps, compare with output 5MP at 5fps, there are more noise of camera data at 15fps case. My debug found is, this noise seems come from ov5640 itself.   Reference: a>https://www.nxp.com/webapp/Download?colCode=IMX8DQXPRM b>https://www.nxp.com/webapp/Download?colCode=L4.14.98_2.0.0_MX8QXP&appType=license c>https://github.com/NXPmicro/gtec-demo-framework d>ov5640 data sheet e>ov5640 software application note f>Efficient, High-Quality Bayer Demosaic Filtering on GPUs https://www.semanticscholar.org/paper/Efficient%2C-High-Quality-Bayer-Demosaic-Filtering-on-McGuire/088a2f47b7ab99c78d41623bdfaf4acdb02358fb

This doc show: on i.MX6Q SabreSD board, configure ov5640 sensor(parallel or MIPI) output 5MP(2592x1944) RAW(Bayer) data at 15fps,and i.MX6Q IPU capture RAW RGB data, and i.MX6Q GPU debayer RAW data then display image. HW: i.MX6Q-SabreSD board, ov5640 sensor. SW: Linux 4.14.98_2.0.0 BSP, and patches in this doc. Configure at camera sensor side A Bayer filter is a color filter array (CFA) for arranging RGB color filters on a square grid of photosensors. The filter pattern is 50% green, 25% red and 25% blue, hence is also called BGGR, RGBG ,GRGB, or RGGB. The ov5640 has an image array capable of operating at up to 15 fps in 5 megapixel (2592x1944) resolution. OV5640 support output formats: RAW(Bayer), RGB565/555/444,CCIR656, YUV422/420, YCbCr422, and compression. To make ov5640 output 5MP RAW data at 15fps, check my patch imx6_ov5640_dvp_mipi_raw_capture_driver-4.14.98_2.0.0.diff which apply on i.MX Linux 4.14.98_2.0.0 BSP kernel code: Parallel interface ov5640, use ov5640_raw_setting[] array of drivers/media/platform/mxc/capture/ov5640.c. This register setting is come from ov5640 software application note and data sheet. MIPI interface ov5640, use ov5640_mipi_raw_setting[] array of drivers/media/platform/mxc/capture/ov5640_mipi.c. This register setting is combine setting of original code (remove ISP register setting), plus PLL register setting for MIPI interface, plus some data format register setting. Configure at i.MX6Q side The i.MX6Q IPU camera port(CSI-2 module) support data format include Raw(Bayer), RGB, YUV 4:4:4, YUV 4:2:2 and grayscale, up to 16 bits per value. Below is camera data routing for i.MX6Q:    Below is i.MX6Q IPU block daigram: The CSI-2 of IPU which is responsible for synchronizing and packing the video (or generic data) and sending it to other blocks. The video data received by CSI-2, could be sent to three other blocks: SMFC, VDI, IC. For RAW (Bayer) data capture, should go through path like this: CSI-2-->SMFC-->IDMAC-->DDR memory It means RAW data is received as generic data, see IPU_PIX_FMT_GENERIC in my patch, and IPU cannot process this kind data, it is just received to DDR memory. For MIPI interface camera, need note is i.MX6 side MIPI D-PHY clock must be calibrated to the actual clock range of the camera sensor’s D-PHY clock and the calibrated value must be equal to or greater than the camera sensor clock, detail see  AN5305. Take MIPI ov5640 as example: Pixel clock = 2592x1944x15fpsx(1/2 cycle/pixel)x1.35 blank interval = 51MHZ MIPI data rate = 51MHZ x 16 bit = 816Mb/s so 816/2/2*2 is 408MHZ is i.MX6 side D-PHY clock. Here due to one bayer pixel is 8bit, and i.MX6 MIPI data bus is 16 bit, so above use 1/2 cycle/pixel. And check ov5640_mipi_raw_setting[], you will got the sensor side D-PHY clock is about 672/2 = 336MHZ. And check AN5305, register MIPI_CSI2_PHY_TST_CTRL1 of i.MX6 need set as 0xC, but here i still keep it as default BSP value 0x14.   3.Capture test code I changed unit test mxc_v4l2_capture.c to capture the RAW data and save it to file. Check my patch imx6_ov5640_raw_captupre_test_4.14.98_2.0.0_ga.diff which apply on i.MX  Linux 4.14.98_2.0.0 BSP unit test code. Note the usage is: ./cap.out -c 1 -i 1 -fr 15 -m 6 -iw 2592 -ih 1944 -ow 2592 -oh 1944 -f BA81 -d /dev/video1 savefile.dmp parameter -i 1 means use CSI to MEM mode /dev/video1 is MIPI ov5640, /dev/video0 is parallel ov5640   4.Display RAW data The RAW data cannot be displayed directly, debayer process is needed to get complete red, green, blue color for each pixel. The debayer process if run on CPU, will cost much CPU time. To save CPU time, debayer could done by GPU. The method is, captured RAW data upload to GPU as texture , then GPU will do the debayer, then full color of each pixel will be got, then display it. To upload RAW camera data to GPU with zero memory copy, i will use i.MX6Q GPU extension GL_VIV_direct_texture. It create a texture with direct access support. API glTexDirectVIVMap,  which support mapping a user space memory or a physical address into the texture surface. The API glTexDirectVIVMap need logic and physical address of data buffer, so i will allocate data buffer from /dev/mxc_ipu, it is dma-buffer also get logic/physical address of buffer, then queue it as USERPTR to ipu v4l2 capture driver, after dequeue got RAW camera data, pass it to GPU for debayer. GPU side, I will use OpenGL shader code from "Efficient, High-Quality Bayer Demosaic Filtering on GPUs". Check my patch imx6-5640-debayer-testcode-gpusdk-5.2.0.diff which apply on i.MX GPU SDK 5.2.0 code. Note, here i only do is debayer, no extra process.   Known issue One thing is ov5640 output 5MP at 15fps, compare with output 5MP at 5fps, there are more noise of camera data at 15fps case. My debug found is , this noise seems come from ov5640 itself.   Reference: a>https://www.nxp.com/webapp/Download?colCode=IMX6DQRM b>https://www.nxp.com/webapp/Download?colCode=L4.14.98_2.0.0_MX6QDLSOLOX&appType=license c>https://github.com/NXPmicro/gtec-demo-framework d>https://www.nxp.com/docs/en/application-note/AN5305.pdf e>ov5640 data sheet f>ov5640 software application note g>Efficient, High-Quality Bayer Demosaic Filtering on GPUs https://www.semanticscholar.org/paper/Efficient%2C-High-Quality-Bayer-Demosaic-Filtering-on-McGuire/088a2f47b7ab99c78d41623bdfaf4acdb02358fb

The NXP i.MX 8M provides industry leading audio, voice and video processing for applications that scale from consumer home audio to industrial building automation and mobile computers. The i.MX 8M Quad supports multiple audio interfaces as listed below: Besides the general audio input/output function, the audio interfaces will supports following features: - SAI-1 supports up to 16-channels TX (8 lanes) and 16-channels RX (8 lanes) at 384KHz/32-bit. - SAI-5 supports up to 8-channels TX (4 lanes) and 8-channels RX (4 lanes) at 384KHz/32-bit. - SAI-2/3/6 supports up to 2-channels TX (1 lanes) and 2-channels RX (1 lanes) at 384KHz/32-bit. - SAI-2/3/6 support up to 2-channels TX (1 lane) and 2-channels RX (1 lane) at 384KHz/32-bit. - SAI-1 supports glue-less switching between PCM & DSD operation for popular audio DACs - SPDIF-1/2 supports raw capture mode that can save all the incoming bits into audio buffer The SAI-1/2/3/5/6 and SPDIF-1 share GPIO pads on the chip through IOMUX. Common use cases supported by the audio interfaces are listed in the table below (many other configurations are possible). The number is the data lanes supported. For the MCLK pin on each SAI module, it can be configured as either input or output. When configured as output, the SAI_CLK_ROOT from CCM will be routed to the pad output. When configured as input, the external input to the pad will be routed to SAI.MCLK, which can be used as master clock for SAI. Below is the diagram showing the both input/output options, by using SAI1 as the example. Each SAI module supports up to 3 master clock inputs. The TX and RX sub-module inside each SAI can independently select one of the clock inputs as its master clock. This allows TX and RX of one SAI to run from different clock source. The master clock inputs have following options: - SAI.MCLK[1] can be selected from SAI_CLK_ROOT from CCM or SAI.MCLK from IOMUX. This is the most straight-forward clock routing in which SAI only use its own clock source from CCM or IO pad. - SAI.MCLK[2] can be selected from following clock sources:       Any of the SAI_CLK_ROOT from CCM;       Any of the SAI.MCLK from IOMUX;       Other clock sources from SPIDF; - SAI.MCLK[3] has exact same clock source options as SAI.MCLK[2]. This allows both TX and RX can have access to all the options without any dependency between each other. The clock options for master clock on SAI are shown in the diagram blow, by using SAI-1 as an example. The options on MCLK[1] is also available on MCLK[2] and MCLK[3]. The reason to keep this options is to provide the similar SAI clock structure as i.MX6/i.MX7 processors. The configuration of the MUX for master clock are controlled by IOMUXC_GPR registers. They should be configured before SAI clock is enabled to avoid glitches on the clock. Note: Because those MUX on clocks are missed during the design, the actual implementation in the silicon is simplified as shown in the following diagram. All the SAI and SPDIF instances have SDMA support. In order to meet the audio data rate, two SDMA modules are used. Because the SAI-2/3 and SPDIF-1/2 do not require high data throughput, they are assigned to SDMA-1, shared with other peripherals such as UART/SPI. SAI-1/4/5/6 need to support high sample rate & multichannel audio, they are assigned to SDMA-2, which is a dedicated SDMA engine for audio. The SDMA-2 frequency is increased to 500/250 instead of 133/66 to make sure it has enough throughput. In order to allow SW tracking the progress of audio DMA, the TX_SYNC and RX_SYNC of SAI modules are routed to GPT as the external clock input. Since there are totally 6 SAI modules, these signals will be MUXed when connection to GPT. - GPT-4/5/6 external clock input can be selected from the TX_SYNC or RX_SYNC of any 6 SAI modules; - The MUX select is controlled by IOMUXC_GPR register; - The MUX select register for GPT-4/5/6 are fully independent of each other.

Platform: i.mx8qm/qxp OS: imx-yocto-L4.14.98_2.0.0_ga Camera: max9286 deserializer <=> max96705 serializer  + ar0144 or: max9286 deserializer <=> max96705 serializer + ov9284 Note that currently only one camera is support and the serializer should be connected to the IN0 of max9286. Data format: ar0144: mono raw 12bit. ov9284: mono raw 10bit. On imx8qm/qxp the data will be recieved as raw 16bit and the valid data bit start from bit[13] to LSB. for mono raw 12bit the valid data bit is 0bxxdd_dddd_dddd_ddxx for mono raw 10bit the valid data bit is 0bxxdd_dddd_dddd_xxxx max9286 and max96705 configuration: dbl bws PXL_CRC/edc hven hibw lccen him should be the same on both sides, this can be achieved by pin or register configurations. The crossbar function of max96705 can be used to fix the reversed data bit. for example, reversed 12bit with dbl to 1. 0x20 0xb 0x21 0xa 0x22 0x9 ....... 0x2b 0x0 0x30 0xb 0x31 0xa .... 0x3b 0x0 0x20 to 0x2b and 0x30 to 0x3b are the registers of max96705. Patch apply: 1. push the kernel-patch to the kernel source and apply it. 2. reconfig the kernel setting, make sure there is only CONFIG_MAX9286_AR0144 or        CONFIG_MAX9286_WISSEN(ov9284) enabled, all other max9286 related are disabled. You can run menuconfig to achieve this. 3. For testing copy the vulkan-v4l2.tar to the board, and run vulkan-v4l2.     the source code is at https://github.com/sheeaza/vulkan-v4l2 branch ar0144 for ar0144, branch ov9284 for ov9284. =========== updated patch for data format.

   Recently, some customers encountered the problem that compilation failed when compiling l4.14.98-2.0.0 fsl-imx-waylan + fsl-image-qt5-validation-imx in Ubuntu 18.04 environment. In fact, compiling QT image is a very time-consuming process, especially in the process of compiling, errors need to be handled, which will be more time-consuming. The following compilation took four days to complete. 1. Environment Linux Host : ubuntu 18.04 LTS Virtual Machine: VMware workstatin Player 12 images: fsl-imx-waylan + fsl-image-qt5-validation-imx Hardware: imx8mqevk Linux BSP verison: L4.14.98-2.0.0 2. Steps (1)Installation of Ubuntu 18.04 2.Update software 3. Installing software package for compiling BSP # sudo apt-get install flex # sudo apt-get install bison # sudo apt-get install gperf # sudo apt-get install build-essential # sudo apt-get install zlib1g-dev # sudo apt-get install lib32ncurses5-dev # sudo apt-get install x11proto-core-dev # sudo apt-get install libx11-dev # sudo apt-get install lib32z1-dev # sudo apt-get install libgl1-mesa-dev # sudo apt-get install tofrodos # sudo apt-get install python-markdown # sudo apt-get install libxml2-utils # sudo apt-get install xsltproc          # sudo apt-get install uuid-dev:i386 liblzo2-dev:i386 # sudo apt-get install gcc-multilib g++-multilib # sudo apt-get install subversion # sudo apt-get install openssh-server openssh-client # sudo apt-get install uuid uuid-dev # sudo apt-get install zlib1g-dev liblz-dev # sudo apt-get install liblzo2-2 liblzo2-dev # sudo apt-get install lzop # sudo apt-get install git-core curl # sudo apt-get install u-boot-tools # sudo apt-get install mtd-utils # sudo apt-get install android-tools-fsutils # sudo apt-get install openjdk-8-jdk # sudo apt-get install device-tree-compiler # sudo apt-get install aptitude # sudo aptitude install libcurl4-openssl-dev nss-updatedb   From i.MX_Yocto_Project_User's_Guide.pdf: # sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \ build-essential chrpath socat libsdl1.2-dev   4. Downloading Yocto BSP according to steps in i.MX_Yocto_Project_User's_Guide.pdf 5．Compiling L4.14.98-2.0.0 BSP # cd ~/imx-yocto-bsp # DISTRO=fsl-imx-wayland MACHINE=imx8mqevk source fsl-setup-release.sh -b build-wayland # bitbake fsl-image-qt5-validation-imx In the process of compilation, there have been many "fetch errors", which are caused by disconnection or timeout of network connection. We just need to run the bitmake command again in the build Wayland subdirectory to continue the compilation. # bitbake fsl-image-qt5-validation-imx          Fetching errors below were what I encountered:          The following picture is to re-run “bitbake fsl-image-qt5-validation-imx” after fetch errors occurred.          In order to improve the speed of compilation , I re-configured vmware player, assigning 6 CPU cores for Ubuntu.          Compilation is a long and arduous process. It took 4 days to compile normally with error handling. Finally, the compilation was completed. NXP TIC Team Weidong Sun 2019-11-02

Most i.MX8QXP/QM customers already work on L4.14.98 GA for their Auto product, like C-V2X TBOX, Car infortainment system. Some customers also want to adopt OP-TEE in their security design, but OP-TEE on i.MX8QXP/QM platform don't support HW cryptography accelerating which base on CAAM module. So I worked on the issue last week and fixed it. The package meta-optee-add-on_4.14.98_2.0.0_ga.tgz is Yocto layer which includes all patches for fixing the issue. Software environments as the belows: Linux kernel: imx_4.14.98_2.0.0_ga HW platform:  i.MX8QM/QXP MEK. How to build: 1, decompress meta-optee-add-on_4.14.98_2.0.0_ga.tgz and copy meta-optee-add-on to folder (Yocto 4.14.98_2.0.0_ga dir)/sources/ 2, Run DISTRO=fsl-imx-wayland MACHINE=imx8qxpmek source fsl-setup-release.sh -b build-optee and add BBLAYERS += " ${BSPDIR}/sources/meta-optee-add-on " into (Yocto 4.14.98_2.0.0_ga dir)/build-optee/conf/bblayers.conf  3, Run bitbake fsl-image-validation-imx. 4, You can run xtest or xtest -l 1 4007 on your MEK board to test optee crypto feature after completing build image. You can find it only take about one second comparing no CAAM accelerating when test "regression_4007.11 Generate RSA-2048 key".

Hello everyone, this document will explain on how to use the UUU (Universal Update Utility) tool to flash Linux to an i.MX device (i.MX 8MM).   Requirements:   MX 8M Mini EVK UUU tool documentation, available here Linux Binary Demo Files - i.MX 8MMini EVK UUU 1.2.135 binary Serial console emulator (tera term or putty)   UUU auto script For this example is used the L4.14.98_2.0.0_ga demo image for the i.MX 8MM, inside the demo image we will find the auto script, which by default flash the eMMC of the board, the structure of the script is as following   /***********************************************************************************/ uuu_version 1.2.39   # This command will be run when i.MX6/7 i.MX8MM, i.MX8MQ SDP: boot -f imx-boot-imx8mmevk-sd.bin-flash_evk   # This command will be run when ROM support stream mode # i.MX8QXP, i.MX8QM SDPS: boot -f imx-boot-imx8mmevk-sd.bin-flash_evk   # These commands will be run when use SPL and will be skipped if no spl # SDPU will be deprecated. please use SDPV instead of SDPU # { SDPU: delay 1000 SDPU: write -f imx-boot-imx8mmevk-sd.bin-flash_evk -offset 0x57c00 SDPU: jump # }   # These commands will be run when use SPL and will be skipped if no spl # if (SPL support SDPV) # { SDPV: delay 1000 SDPV: write -f imx-boot-imx8mmevk-sd.bin-flash_evk -skipspl SDPV: jump # }   FB: ucmd setenv fastboot_dev mmc FB: ucmd setenv mmcdev ${emmc_dev} FB: ucmd mmc dev ${emmc_dev} FB: flash -raw2sparse all fsl-image-validation-imx-imx8mmevk.sdcard FB: flash bootloader imx-boot-imx8mmevk-sd.bin-flash_evk FB: ucmd if env exists emmc_ack; then ; else setenv emmc_ack 0; fi; FB: ucmd mmc partconf ${emmc_dev} ${emmc_ack} 1 0 FB: done /***********************************************************************************/    In short, when the board goes into serial downloader mode UUU downloads the bootloader to internal RAM, once done and uboot is running, through fastboot utility it will flash .sdcard file and uboot to the eMMC on the board.   More information about the protocol UUU use please refer to the UUU documentation (UUU.pdf) section 5 Supported protocol.   Running the tool In order to run the tool the binary of uuu needs to be downloaded, the binary files can be downloaded from the link above, uuu.exe is for Windows and uuu is for Linux. Once downloaded it can be placed inside the same file as the demo image, this so it is easy to run and cleaner on the shell commands.   Windows In windows OS the tool should be run using the Windows PowerShell in administrator mode, once open we will run the next commands: > .\uuu.exe uuu.auto   Linux >$ sudo ./uuu uuu.auto   The tool will start running and should be waiting for any i.MX device to be detected by host pc   Preparing the board For the board to be flashed it is needed to be in download mode, the switch configuration (i.MX 8MM EVK) is as following: SW1101  -  1010XXXXXX SW1102  -  XXXXXXXXX0   Connect a USB cable from the host pc which will run the tool to the USB OTG/TYPE C port, usually specified as download, on the board.   Connect a USB cable from the host to the OTG-to-UART for console output, usually specified as debug, on the board.   Open terminal emulator program with the following settings: Bits per second - 115200 Data bits - 8 Parity - None Stop bits - 1 Flow control - None   Power on the board, the download will start and the serial prompt will show the progress in uboot, wait until the tool show success.   Finally power off the board and change the switch configuration to boot from the eMMC, power on the board again and it should boot successfully!   Built in scripts One can use the built in scripts using the -b option to burn the bootloader  and the rootfs to the target flash, just type the command accordingly to the target flash device.    SD Write bootloader only: Windows: > .\uuu.exe -b sd <bootloader> Linux: $ sudo ./uuu -b sd <bootloader>   Replace <bootloader> for your .imx/.bin file, example using the i.MX 8MM for Windows and Linux respectively below. > .\uur.exe -b sd imx-boot-imx8mmevk-sd.bin-flash_evk $ sudo ./uuu -b sd imx-boot-imx8mmevk-sd.bin-flash_evk    Write whole Linux image Windows: > .\uuu.exe -b sd_all <bootloader> <rootfs>.sdcard Linux: $ sudo ./uuu -b sd_all <bootloader> <rootfs>.sdcard   Replace <bootloader> and <rootfs> for the name of your .imx/.bin and .sdcard files respectively, example using the i.MX 8MM below. > .\uuu.exe -b sd_all  imx-boot-imx8mmevk-sd.bin-flash_evk fsl-image-validation-imx-imx8mmevk.sdcard $ sudo ./uuu -b sd_all  imx-boot-imx8mmevk-sd.bin-flash_evk fsl-image-validation-imx-imx8mmevk.sdcard   eMMC Write bootloader only Windows: > .\uuu.exe -b emmc <bootloader> Linux: $ sudo ./uuu -b emmc <bootloader>   Example using i.MX 8MM > .\uuu.exe -b emmc imx-boot-imx8mmevk-sd.bin-flash_evk $ sudo ./uuu -b emmc imx-boot-imx8mmevk-sd.bin-flash_evk   Write whole Linux image Windows: > .\uuu.exe -b emmc_all <bootloader> <rootfs>.sdcard Linux: $ sudo ./uuu -b emmc_all <bootloader> <rootfs>.sdcard   Example using i.MX 8MM > .\uuu.exe -b emmc_all imx-boot-imx8mmevk-sd.bin-flash_evk fsl-image-validation-imx-imx8mmevk.sdcard $ sudo ./uuu -b emmc_all imx-boot-imx8mmevk-sd.bin-flash_evk fsl-image-validation-imx-imx8mmevk.sdcard   Hope this will helpful for everyone who is starting to use this flashing tool.

For the board imx8M Quad EVK running the Linux 4.14.78-1.0.0_ga version BSP, the resolutions 3840x2160,1920x1080, 1280x720, 720x480 are support in our default BSP. For the other resolutions how to make it work? This patch used to do support for a non-default resolution on i.MX 8MQ EVK. Basically, the customer needs to change the clocks accordingly to the display requirements,  it to be used as a base to the display support.

KSZ9031 is a very common PHY used with many ethernet design. This document will show you how to add it in u-boot and kernel. 1. Schematic The MODE[3:0] strap-in pins are sampled and latched at power-up/reset. MODE[3:0]=1111 is RGMII mode - Advertise all capabilities (10/100/1000 speed half-/full-duplex) The PHY address, PHYAD[2:0], is sampled and latched at power-up/reset. Here PHY address is set to 001. In this design example, the ENET_RESET_B is connected to GPIO pin GPIO1_IO03. 2. Source code modification In u-boot source code, add the following code in the <board_name>.c file. - IOMUX setup for the GPIO1_IO03 pin. static iomux_v3_cfg_t const phy_reset_pads[] = {      MX7D_PAD_GPIO1_IO03__GPIO1_IO3 | MUX_PAD_CTRL(NO_PAD_CTRL), }; - In the function setup_fec(int fec_id), add the code for phy reset. imx_iomux_v3_setup_multiple_pads(phy_reset_pads, ARRAY_SIZE(phy_reset_pads)); gpio_request(IMX_GPIO_NR(1, 3), "ENET PHY Reset"); gpio_direction_output(IMX_GPIO_NR(1, 3) , 0); mdelay(20); gpio_set_value(IMX_GPIO_NR(1, 3), 1); - There is a PHY config for the KSZ9031. int board_phy_config(struct phy_device *phydev) {    /*      * Default setting for GMII Clock Pad Skew Register 0x1EF:      * MMD Address 0x2h, Register 0x8h      *      * GTX_CLK Pad Skew 0xF -> 0.9 nsec skew      * RX_CLK Pad Skew 0xF -> 0.9 nsec skew      *      * Adjustment -> write 0x3FF:      * GTX_CLK Pad Skew 0x1F -> 1.8 nsec skew      * RX_CLK Pad Skew 0x1F -> 1.8 nsec skew      *      */     /* control data pad skew - devaddr = 0x02, register = 0x04 */     ksz9031_phy_extended_write(phydev, 0x02,                    MII_KSZ9031_EXT_RGMII_CTRL_SIG_SKEW,                    MII_KSZ9031_MOD_DATA_NO_POST_INC, 0x0000);     /* rx data pad skew - devaddr = 0x02, register = 0x05 */     ksz9031_phy_extended_write(phydev, 0x02,                    MII_KSZ9031_EXT_RGMII_RX_DATA_SKEW,                    MII_KSZ9031_MOD_DATA_NO_POST_INC, 0x0000);     /* tx data pad skew - devaddr = 0x02, register = 0x05 */     ksz9031_phy_extended_write(phydev, 0x02,                    MII_KSZ9031_EXT_RGMII_TX_DATA_SKEW,                    MII_KSZ9031_MOD_DATA_NO_POST_INC, 0x0000);     /* gtx and rx clock pad skew - devaddr = 0x02, register = 0x08 */     ksz9031_phy_extended_write(phydev, 0x02,                    MII_KSZ9031_EXT_RGMII_CLOCK_SKEW,                    MII_KSZ9031_MOD_DATA_NO_POST_INC, 0x03FF);     if (phydev->drv->config)         phydev->drv->config(phydev);     return 0; } The KSZ9031 driver (drivers/net/phy/micrel.c) had already supported in the u-boot source code. Add the following #define in the <board_name>.h file to enable the driver for building. #define CONFIG_PHY_MICREL As the PHY address on the board is 001, change the PHYADDR to 1 in the <board_name>.h file. #define CONFIG_FEC_MXC_PHYADDR          0x1 In the kernel source code, add/modify the PHY setting in dts file like this. &fec1 {     pinctrl-names = "default";     pinctrl-0 = <&pinctrl_enet1 &pinctrl_enet_reset>;     assigned-clocks = <&clks IMX7D_ENET_PHY_REF_ROOT_SRC>,               <&clks IMX7D_ENET_AXI_ROOT_SRC>,               <&clks IMX7D_ENET1_TIME_ROOT_SRC>,               <&clks IMX7D_ENET1_TIME_ROOT_CLK>,               <&clks IMX7D_ENET_AXI_ROOT_CLK>;     assigned-clock-parents = <&clks IMX7D_PLL_ENET_MAIN_25M_CLK>,                  <&clks IMX7D_PLL_ENET_MAIN_250M_CLK>,                  <&clks IMX7D_PLL_ENET_MAIN_100M_CLK>;     assigned-clock-rates = <0>, <0>, <0>, <100000000>, <250000000>;     phy-mode = "rgmii";     phy-handle = <&ethphy0>;     phy-reset-gpios = <&gpio1 3 0>;     fsl,magic-packet;     status = "okay";     mdio {         #address-cells = <1>;         #size-cells = <0>;         ethphy0: ethernet-phy@1 {    //here '@1' is the PHY address             compatible = "ethernet-phy-ieee802.3-c22";             reg = <1>;         };     }; }; Add the GPIO pin for the ENET_RESET_B &iomuxc { ... ... pinctrl_enet_reset: enet_resetgrp {             fsl,pins = <                 MX7D_PAD_GPIO1_IO03__GPIO1_IO3      0x14     //ENET_RESET_B             >;         }; } There is a PHY fixup in the arch/arm/mach-imx/<imx_cpu>.c Here is the example in mach-imx7d.c #define PHY_ID_KSZ9031    0x00221620 #define MICREL_PHY_ID_MASK 0x00fffff0 static void mmd_write_reg(struct phy_device *dev, int device, int reg, int val) {     phy_write(dev, 0x0d, device);     phy_write(dev, 0x0e, reg);     phy_write(dev, 0x0d, (1 << 14) | device);     phy_write(dev, 0x0e, val); } static int ksz9031rn_phy_fixup(struct phy_device *dev) {     /*      * min rx data delay, max rx/tx clock delay,      * min rx/tx control delay      */     mmd_write_reg(dev, -1, 0x4, 0);     mmd_write_reg(dev, -1, 0x5, 0);     mmd_write_reg(dev, -1, 0x6, 0);     mmd_write_reg(dev, -1, 0x8, 0x003ff);     return 0; } static void __init imx7d_enet_phy_init(void) {     if (IS_BUILTIN(CONFIG_PHYLIB)) {         phy_register_fixup_for_uid(PHY_ID_AR8031, 0xffffffff,                        ar8031_phy_fixup);         phy_register_fixup_for_uid(PHY_ID_BCM54220, 0xffffffff,                        bcm54220_phy_fixup);         phy_register_fixup_for_uid(PHY_ID_KSZ9031, MICREL_PHY_ID_MASK,                 ksz9031rn_phy_fixup);     } } Now, the PHY is working on your board. Reference: 1.  Create an Ubuntu VM environment to build Yocto BSP  2.  i.MX Software | NXP