This PDF is training material for showing examples on video encoding, video decoding, video streaming on an i.MX53QSB board.

[中文翻译版] 见附件   原文链接： https://community.nxp.com/docs/DOC-343054 

We will cross compile L5.4.3_1.0.0 on ubuntu18.04 LTS platform in three steps. 1. Installing Ubuntu 18.04 to vmplayer14 & Configuring Ubuntu 18.04 LTS 2. Compiling L5.4.3_1.0.0 BSP According to Steps in i.MX_Yocto_Project_User's_Guide.pdf 3. Handling tensorflow-lite compilation errors. --BEGIN No.1 Installing Ubuntu 18.04 to vmplayer14 & Configuring Ubuntu 18.04 LTS 1. Install 18.04 on vmplayer After installation is done, root user should be set at first. # sudo passwd root Then follow these steps to configuration ubuntu 18.04 for environment of compiliation --Changing sources of ubuntu 18.04 mirror If you are Chinese users, you can do the step, which can improve your system performance. # sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak # sudo geit /etc/apt/source.list Comment I.MX customers outside China do not need to modify Ubuntu source list, or can modify it to local mirror site of Ubuntu 18.04, which can improve the speed of software upgrade. Delet all sources and copy following lines here, Then save it and exit Update ubuntu source deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse # deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse Then running these 2 commands to update sources and packages # sudo apt-get update # sudo apt-get upgrade 2. Installing packages for compiliation Packages For Compiling BSP # sudo apt-get install flex bison gperf build-essential zlib1g-dev lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z1-dev libgl1-mesa-dev tofrodos python-markdown libxml2-utils xsltproc # sudo apt-get install uuid-dev:i386 liblzo2-dev:i386 gcc-multilib g++-multilib subversion openssh-server openssh-client uuid uuid-dev zlib1g-dev liblz-dev lzop  liblzo2-2 liblzo2-dev git-core curl # sudo apt-get install u-boot-tools mtd-utils android-tools-fsutils openjdk-8-jdk device-tree-compiler aptitude libcurl4-openssl-dev nss-updatedb # sudo apt-get install chrpath texinfo gawk cpio diffstat No.2 Compiling L5.4.3_1.0.0 BSP According to Steps in i.MX_Yocto_Project_User's_Guide.pdf          The detailed BSP compilation steps are given in i.MX_Yocto_Project_User's_Guide.pdf. Here are just some tips and simple steps. For Chinese customers, the system may not respond when the repo init or repo sync command is executed after obtaining the repo script. Detailed description is below: (1)Getting repo script # mkdir ~/bin (this step may not be needed if the bin folder already exists) # curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo # chmod a+x ~/bin/repo # export PATH=~/bin:$PATH (2)Getting Yocto source # mkdir imx-yocto-bsp # cd imx-yocto-bsp # repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-zeus -m imx-5.4.3-1.0.0.xml # repo sync          After running repo init or repo sync, you may encounter a situation where there is no progress for a long time, and repo sync stops there without any prompt. This may be caused by googlesource site being inaccessible.          At this time, you can modify the googlesource URL to the URL of Tsinghua University. # gedit ./repo          Changing REPO_URL = 'https://gerrit.googlesource.com/git-repo'          To be:                    REPO_URL = 'https://mirrors.tuna.tsinghua.edu.cn/git/git-repo'          Save and exit. Then run “repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-zeus -m imx-5.4.3-1.0.0.xml”. or repo sync No.3 Handling tensorflow-lite compilation errors       If you encounter a tensorflow-lite compilation error during the compilation process, the following processing methods may be helpful to you. Cause Analysis:          Compiling tensorflow-lite requires three steps: fetching packages, configuring environment, and compiling code.          During the configuring environment, the following 8 software packages need to be downloaded. If any of the 8 files are not successfully downloaded, an error will be reported and compilation will be terminated.          ①9f48e814419e.tar.gz          ②38ebac7b059e84692f53e5938f97a9943c120d98.zip          ③release-1.8.0.tar.gz          ④389ec3f906f018661a5308458d623d01f96d7b23.tar.gz          ⑤master.zip          ⑥816a4ae622e964763ca0862d9dbd19324a1eaf45.tar.gz          ⑦1f5eae5d6a135ff6811724f6c57f911d1f46bb15.tar.gz          ⑧fft.tgz          If you encounter this problem, you can refer to the following methods to solve it. Open the file with editor: ~/imx-yocto-bsp/build-xwayland/tmp/work/aarch64-poky-linux/tensorflow-lite/1.13.2-r0/git/tensorflow/lite/tools/make/download_dependencies.sh URLs EIGEN_URL="https://bitbucket.org/eigen/eigen/get/9f48e814419e.tar.gz" GEMMLOWP_URL="https://github.com/google/gemmlowp/archive/38ebac7b059e84692f53e5938f97a9943c120d98.zip" GOOGLETEST_URL="https://github.com/google/googletest/archive/release-1.8.0.tar.gz" ABSL_URL="https://github.com/abseil/abseil-cpp/archive/389ec3f906f018661a5308458d623d01f96d7b23.tar.gz" NEON_2_SSE_URL="https://github.com/intel/ARM_NEON_2_x86_SSE/archive/master.zip" FARMHASH_URL="https://github.com/google/farmhash/archive/816a4ae622e964763ca0862d9dbd19324a1eaf45.tar.gz" FLATBUFFERS_URL="https://github.com/google/flatbuffers/archive/1f5eae5d6a135ff6811724f6c57f911d1f46bb15.tar.gz" FFT2D_URL="http://www.kurims.kyoto-u.ac.jp/~ooura/fft.tgz" Save and exit. [comment]     Don’t run “bitbake tensorflow-lite -c cleanall before you modify it”, otherwise files will be deleted.  Then try to compile tensorflow-lite like below: # bitbake tensorflow-lite -c compile   At the same time, we can open this directory and check if the following directories are created. If these 8 directories are successfully created, it means that links we modified are no problem.          After tensorflow-lite is successfully built, we can continue to compile full images. # bitbake imx-image-full --END NXP TIC team Weidong Sun 2020/3/30

Q: When trying to mount his SDIO WiFi module (an Azurewave module containing a Marvel 88W8790)  in 4 bit mode, and got error wifi module on iMX6 Smart SD dev board using the alpha kernel (3.5.7+3285970). mwifiex_sdio mmc0:0001:1: WLAN FW is active mwifiex_sdio mmc0:0001:1: mwifiex_cmd_timeout_func: Timeout cmd id (2004.239824) = 0xa9, act = 0x0 mwifiex_sdio mmc0:0001:1: num_data_h2c_failure = 0 mwifiex_sdio mmc0:0001:1: num_cmd_h2c_failure = 0 mwifiex_sdio mmc0:0001:1: num_cmd_timeout = 1 mwifiex_sdio mmc0:0001:1: num_tx_timeout = 0 mwifiex_sdio mmc0:0001:1: last_cmd_index = 1 mwifiex_sdio mmc0:0001:1: last_cmd_resp_index = 0 mwifiex_sdio mmc0:0001:1: last_event_index = 0 mwifiex_sdio mmc0:0001:1: data_sent=1 cmd_sent=1 mwifiex_sdio mmc0:0001:1: ps_mode=1 ps_state=0 +++++++++++++++++++++++ There are only two known issues with the SDHC driver as noted in the release notes and these don't seem to match.  The Linux Reference Manual states the operation of the SDIO was veriifed usign the AR6003.  My assumption is that this is the Silex module we have standardized on.  Was 4 bit mode verified using this module? Does anyone have any idea what could be happening here? A: L3.5.7 is an alpha release, and WiFi function is not stable. The test report shows that open WiFi issue exists. Wifi module sometime can't work on mx6q_smd. After insmod ath, ath6kl_core and ath6kl_sdio. Insert the wifi card, sometime(50%) will display -------------------------- ath6kl: unable to read RX_LOOKAHEAD_VALID                                      ath6kl: Unable to recv target info: -84                                        ath6kl: Failed to init ath6kl core                                             ath6kl_sdio: probe of mmc0:0001:1 failed with error -84    -------------------------- reinsert the wifi card may solve the problem. when execute "udhcpc -i wlan0", sometime (30%) will cause program exception. Sometime can't get the wlan0 ip(program hang). Environment(OS,Platform,Driver, etc): HW: MX6Q_SMD Num014 and Num017 MX6Q_ARD don't have this problem SW: Kernel 3.5.7-1.0.0 GNU/Linux Case ID: TGE-LV-WIFI-0043 Reproduce Steps: #modprobe ath #modprobe ath6kl_core #modprobe ath6kl_sdio insert the wifi card # iwconfig wlan0 mode managed # iwlist wlan0 scanning | grep MAD-wifi #iwconfig wlan0 key 00112233445566778899123456 #iwconfig wlan0 essid MAD-wifi #udhcpc -i wlan0 Attached MX6 ARD WiFi issue also. [Kernel3.5.7_MX6QARD]Wifi:wifi card can't work during suspend and resume. 100% -- Bug detailed description: With wifi card inserted in the board. doing system suspend and resume test. wifi card can't work well after the system suspend and resume. Always report : ath6kl: Unable to decrement the command credit count register: -84             ath6kl: Unable to write to the device: -84                                     ath6kl: bmi_write_memory for uart debug failed                                 ath6kl: Failed to boot hw in resume: -5          Environment(OS,Platform,Driver, etc): HW: MX6QARD -023 Only tried on this platform SW: root@imx6qsabreauto:~# uname -a                                                Linux imx6qsabreauto 3.5.7-1.0.0+3285970 #1 SMP PREEMPT Sat Jun 29 10:20:45 CDT 2013 armv7l GNU/Linux Case ID:  TGE-LV-WIFI-1060 and TGE-LV-WIFI-1062 Reproduce Steps: 1. boot the kernel with wif card inserted 2. doing wifi stress test 3. doing suspend and resume

i.MX6X_内核驱动代码与定制_V2-20150518.doc: 3.0.35

Hello Linux and i.MX6 users Tushar has posted on the Element14 community some clear tutorials (accessible following the link below) to create a bootable SD-Card from a Linux image. Linux host:             http://www.element14.com/community/community/designcenter/single-board-computers/riotboard/blog/2014/07/31/upgrading-riotboard-with-kernel-31017-linux-host Windows host:      RIoTboard: Upgrading RIoTboard with Kernel 3.10... | element14 Happy SD-Card Programming Greg

There are two optionsou 1. Remove all source code after building: You can indicate bitbake to remove all source code after a build adding this line INHERIT += "rm_work" our to your conf/local.conf file. Of course, when baking another image, ALL source code has to be fetched again and it can take a lot of time, specially on machines with low connections to the Internet. The smallest image you can build is core-image-minimal, containing the minimal set of packages to board a particular machine. The largest is fsl-image-gui (in fact, there is one bigger 'fsl-image-gui-sdk' used for developing purposes inside the target). Between these two, there are a lot of images you can choose. You can even create your own image using hob. In case you do not want to build anything, there are some pre-built images on this site, just download the sdcard file and flash it to an SD card. 2. Select only those  IMAGE_FSTYPES you need. There is a variable named IMAGE_FSTYPES  which indicates the output formats you want for the resulting root file system images, by default is has this value: IMAGE_FSTYPES ?= "tar.bz2 ext3 sdcard" and it is located on /sources/meta-fsl-arm/conf/machine/include/imx-base.inc machine's header file. If you want to change it, place this variable on the build/conf/local.conf file. In case you have NFS mounting, remove the 'sdcard' (and either 'ext3' or 'tar.bz2') string, resulting in IMAGE_FSTYPES = "tar.bz2"

In the i.MX51 default WINCE6  release, the eCSPI doesn't support multiple bursts mode and set the wait states. Attached was the document and code for how to enable the multiple bursts mode and how to set the wait states between two burst.

We have a ATK tool which can program image, also it can burn fuse for i.MX51. Since fuse is one time program, so please take care the fuse can't be turn back after programmed.

Very often, customers need to disable the framebuffer auto blank. There are several ways to do so. Modify the code in drivers/tty/vt/vt.c or remove the  “CONFIG_VT_CONSOLE” from the kernel configuration. All these work, but inconvenient. Review the code following: static int vesa_blank_mode; /* 0:none 1:suspendV 2:suspendH 3:powerdown */ static int vesa_off_interval; static int blankinterval = 10*60; core_param(consoleblank, blankinterval, int, 0444); The blank is controlled by blankinterval, which can be set with the name consoleblank. And the consoleblank is a "core_param". core_param in linux can be recognized by kernel. Also can be passed to kernel command line from uboot with bootargs. we could add this to the bootargs that the framebuffer will not go blank: consoleblank=0 Example(verified with imx6 L3.0.35_4.1.0_130816): setenv bootargs_mmc 'setenv bootargs ${bootargs} root=/dev/mmcblk1p1 rootfstype=ext4 rootwait video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24,bpp=32 fbmem=28M consoleblank=0'

View Webinar Recording

Brief introduction on the aarch64 linux kernel memory mapping layout and basic management stuffs.  Contents include: Kernel's virtual memory layout and mapping after running i.MX8QM/QXP kernel reserved memory layout Kernel memory allocation method and technology (Buddy, cma, ION...) DMA buffer management, SWIOTLB, IOMMU GPU memory management How to customize the memory for different use cases How to avoid using CMA for a better stability and performance

These questions and answers are about interrupt generation at a dedicated (configurable) video output port. The i.MX6D manual (Rev. 0) Image Processing Unit (IPU) chapter mentions: Every DI has 10 timing generator counters. The IPU Interrupt Generator has 10 DI0 counters (1...10) and just 2 DI1 counters (3 & 😎 as interrupt sources. The Interrupt Control Register lists 11 DI0 counters (0...10) Q1. Are the DI timing-generator counters linked to the counters in the interrupt controller, or are they different counters? A1. Yes, the DI timing generator counters are linked to the counters in the interrupt controller. Q2. Why are there 11 counters listed in the interrupt controller, but just 10 counters in the timing generator? A2. There is disp_clk_en_pre in the interrupt controller. Thus the 11 counters: 10 timing generator counters and 1 disp clock generator counter. Q3. Is configurable timing feasible for DI0 by using the timing generator counters? A3. Yes, using the 10 internal timing counters you can generate various timing relationships. In addition, you can detect any of the interrupt counters. For example, if you use counter 8, then you can detect the interrupt associated with counter 8. Q4. Explain the impact of the DI1 counter access of only channels 3 and 8. A4. DI1 also has 10 timing generator counters and 1 disp clock generator counter, which you can use to generate desired waveforms. This is similar to DI0. The difference is only 2 of the 10 counters (plus another disp_clk) are connected to the interrupt controller for DI1. Therefore, there is a restriction for detection. If you use counter 7, read out the counter 7 interrupt of DI1 is not possible. However, 2 channels should be sufficient. These interrupts are usually used to indicate a frame start or a frame end. We usually use counter 3 to represent Vsync. So normally we only use counter 3 interrupt. DI1 has only 3 accesses because this covers the anticipated use case and the desire was to restrict register size. The extra counters facilitate flexible DI1 timing generation.

This doc show how to use i.MX8QXP Display Controller GammaCor unit to tune gamma. HW: i.MX8QXP MEK board, HDMI monitor SW: i.MX Linux 4.14.98_2.2.0 BSP release, patch in this doc 1.Introduce gamma The gamma, gamma correction, gamma encoding, gamma compression , these words all related one kind operation , see wiki page of it: The device used for image capture/print/display follow this power-law. For example the camera captured image , to view this image on display device as good as original captured image : gamma encoding when camera saved sensor data to image file,  and  gamma decoding when that image file display on your PC LCD monitor. That is : 2. i.MX8QXP Display Controller Gamma Correction Unit The Gamma Correction unit position is located between Frame Gen unit and TCon unit.   More detail see below contents from i.MX8QXP RM: So GammaCor unit could be used as adjust display gamma , or brightness or contrast. To used it, need follow the steps at RM 15.9.2.4.4.8.3.   Something need to note: You need program 33 sample point value into the register, these sample point value range is from 0 to 1023. Note, first write is start sample point value , then the other is delta value: current sample point minus previous sample point value. You can use GammaCor unit on any channel of R/G/B. If you use normalized function f(x), the following formula should be used to clut[i = 0..32] = round( f(i * 32 / 1023) * 1023) 3. i.MX8QXP Linux device driver patch and test code Apply attached  patch 8qxp_dpu_gammacor_4.14.98_2.2.0.diff on Linux kernel. In the kernel patch, function dpu_gammacor_update, I choose not calculate delta value between each sample pint , let user space application calculate delta value and passed to kernel. Apply 8qxp-dpu-gammacor-modetst.diff on libdrm-imx, to get test application which is based on modetest.  Test app will read one greyscale image file 720P.rgb, put it under same folder of test application , calculate sample point value by pow function  , and calling drmModeCrtcSetGamma to pass related value to kernel,  next loop will change sample point value, and will see that greyscale image will changed on HDMI monitor. After system boot up, run below cmd to check result of test application systemctl stop weston ./gamma_show_rgba.out -P 29@32:1280x720@AB24 Reference: a>https://www.nxp.com/webapp/Download?colCode=IMX8DQXPRM b>https://www.nxp.com/webapp/Download?colCode=L4.14.98_2.2.0_MX8QXP&appType=license c> https://source.codeaurora.org/external/imx/libdrm-imx/ d> https://en.wikipedia.org/wiki/Gamma_correction

Debugging with Eclipse and GDB on Linux user space This is a good open-source choice to debug i.MX processors. The integration of popular tools like Eclipse and GDB offers a good and stable connection between host and target. The first step is to install the tools on host. Click here to get instructions of how to install the tools. Let's debug a ready hello world program into the ltib. To extract the hello world package, type on ltib directory: $./ltib -m prep -p helloworld Change the code of hello.c to: #include <stdio.h> int main(int argc, char** argv) {     int i;     for (i = 0; i < 100; ++ i)     {         printf(“Welcome to GDB ! %d /n”, i);     }     return 0; } Change the Makefile to add debug symbols Change the two following lines from: CFLAGS = -Wall CXXFLAGS = -Wall To: CFLAGS = -Wall -g CXXFLAGS = -Wall -g Build and deploy the new source-code: $./ltib –p helloworld –m scbuild $./ltib –p helloworld –m scdeploy Configuring the Target On target, gdbserver needs to be run to perform debug. The gdbserver command has the following structure: gdbserver ip_host:port /full/path/app/app_name If gdbserver is not installed on target, select gdb package on ltib configuration. In this example our host has the 192.168.16.35 IP address and our HelloWorld application is located at /usr/bin/hello on the target board. Execute the gdbserver: gdbserver 192.168.16.35:10000 /usr/bin/hello You can use other port number as long as you use the same number when configuring the Eclipse. Setting a GDB Debug Session on Eclipse Now we will configure Eclipse C/C++ to start a GDB session with our remote i.MX board. We will need to know which is the target board’s IP address. To get your target’s IP address: /sbin/ifconfig In our example the target board has the 192.168.16.36 IP address. Open-up Eclipse and choose the C/C++ perspective. We will import the HelloWorld executable built by LTIB. Go to the menu File -> Import You will see the “Import” screen. Select “C/C++ Executable” option. Hit the “Next” button. Eclipse automaticaly creates a new project when whe use the “Import” option. In the next screen, select the “Search Directory” option and hit the “Browse” button. This session is incomplete and is being edited...

Test digital zoom with ipu for camera preview.   Board :sarbre-sd (imx6dq) BSP   : android 13.4ga In the above flow, one frame buffer is processed in four steps at camera preview. Add the step to change the frame buffer before step 4 , the added step which  zoom one preview frame.   The figure below shows the crop function of ipu lib, we use this function scale the frame.   Test result: preview zoom levle 0:   preview zoom level max:     When taking pictures with 5M pixels and the zoom is over level 1, the picture size is not 2592x1944 but 2016x1512. The underlying reason for it is that ipu crop function only supports the 2048x2048 maximum output .   Thumbnails of test result :  

Installation, patching and building the SDK The i.MX 6 Series Platform SDK v1.1.0 does not enable neither the MMU nor L1/L2 Caches (depending on the benchmark you are running, enabling these yield much better numbers), so there is a need to patch the code to enable these. Please download the SDK from the Freescale portal and the patch attached on this document, then follow these instructions: $ tar zxvf imx6_platform_sdk_v1.1.0.tgz $ cd iMX6_Platform_SDK $ patch -p1 < 0001-add-L2-cache-enable-to-mx6-SDK-1.1.0.patch $ export PATH=$PATH:<toolchain_install_path>/bin $ ./tools/build_sdk For more help, please look at the README.pdf and documents inside the doc folder.

[中文翻译版] 见附件   原文链接： https://community.nxp.com/docs/DOC-345644 

Q: How to setup camera under Android? helping a customer (RTX) debug some issues with camera. They're using i.MX 6Solo and Android 13.4-GA on custom hardware. They added a new camera driver which seems to work when using small console capture program (no gui, no preview), so the route from camera to /dev/video0 seems to work. However, when they try to use camera from Android, entire system freezes. They located the crash to following line in ipu_common.c which basically enables camera CSI0: ipu_cm_write(ipu, reg | IPU_CONF_CSI0_EN, IPU_CONF); While investigating IPU setup, we noticed that CPMEM setup for IDMAC channel 0 is "off": ch 0 word 0 - 00000000 25800000 00000000 E0000000 00077C4F ch 0 word 1 - 01B086B0 00394EC0 0087C000 00009FC0 0000027F As seen from above, EBA0 points to 0x0D843580 and EBA1 to 0x0E53B000, which is in EIM memory space, not DDR memory space, which probably causes issues. We're not sure what could be causing this as camera driver doesn't provide any such address and mostly just handles communication to camera chip. But something gets off in Android framework and we could use any hints about what to look for. A: You can reference to the "i.MXAndroidR13.4GAAdvancedUserGuide.html" "3 Camera&Video Recorder customization" for how to change the camera in Android. Another thing needs be checked is the camera sensor driver, you can reference to "kernel_imx\drivers\media\video\mxc\capture\ov5642.c", "static struct v4l2_int_ioctl_desc ov5642_ioctl_desc", did you implemented the same v4l2_int_ioctl_desc functions? Customer HW is set up so that DDR memory space starts at 0x80000000. This causes problems in myandroid/hardware/imx/mx6/libcamera/CamerHAL.cpp when obtaining buffer addresses where camera should store data. GPU will only return OFFSET into 2GB memory space it can address, so it will return an address below 0x80000000. This needs to be adjusted before passing onto V4L2 when starting capturing.