This document is a user guide for the GStreamer version 1.0 based accelerated solution included in all the i.MX 8 family SoCs supported by NXP BSP L5.4.24_1.1.0. Some instructions assume a host machine running a Linux distribution, such as Ubuntu, connected to i.MX 8 device. These commands were tested using Ubuntu 18.04 LTD, and while Ubuntu is not required on the host machine, other distributions have not been tested. These instructions are targeted for use with the following hardware: • i.MX 8MQ EVK • i.MX 8MN EVK • i.MX 8MN EVK • i.MX 8QXP MEK B0 • i.MX 8QM MEK B0   Release History v1.0 - Mar 2020 - Initial release. v2.0 - Sep 2020: Added the following content: - Mux/Demux Examples - Audio Examples - Image Examples - Transcode Examples - Streaming Examples - Multi-Display Examples - Scaling and Rotation Examples - Zero-copy Examples - Debug Examples Maintainers: . Marco Franchi . Pedro Jardim

In order to get USB cameras (web cams) working on i.MX 51 EVK board running Ubuntu, a few steps must be followed, and they are: Enable USB Camera's drivers on Kernel Test it using Gstreamer or another compatible software (as Cheese) Kernel Driver USB cameras (web cameras) on Linux work over GSPCA driver, to enable this driver you need to go to: ./ltib -c   [*] Configure the kernel     Device Drivers -->          Multimedia Devices -->               [*] Video Capture Adapters -->                    [*] V4L USB Devices -->                         <*> USB Video Class (UVC)                                      [*] UVC input events device support                         <*> GSPCA Based WebCams --> From this point, you need to choose your specific driver. If you don't know, you can select all of those options as a built-in module "<*>" that will work. GSPCA Drivers USB Camera Detection Connect your USB camera to the USB Host port on i.MX 51 EVK board and then type "dmesg", and also check if there is a video0 device using: ubuntu@ubuntu-desktop:~$ ls /dev/video0 /dev/video0 USB Camera Detection Gstreamer Command Line In order to test your USB camera using Gstreamer plugin, use the following command line to perform it: ubuntu@ubuntu-desktop:~$ gst-launch-0.10 v4l2src ! ffmpegcolorspace ! ximagesink and the results: Hi there !!! EOF !

The i.MX 8QXP MEK does not allow the OV5640/LVDS/LCD usage only by changing the device tree anymore. It occurs because the M4 owns the i2c resources, so the A core must use rpmsg to enable virtual drivers. Due to this, if the user changes the device tree, for instance, the *ov5640.dtb, the kernel won't boot, entering in the following loop: [    8.603353] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).      [    8.610025] [drm] No driver support for vblank timestamp query.              [    8.616077] imx-drm display-subsystem: bound imx-drm-dpu-bliteng.2 (ops dpu_) [    8.624978] imx-drm display-subsystem: bound imx-dpu-crtc.0 (ops dpu_crtc_op) [    8.632526] imx-drm display-subsystem: bound imx-dpu-crtc.1 (ops dpu_crtc_op) [    8.639833] imx-drm display-subsystem: failed to bind ldb@562210e0 (ops imx_7 [    8.648428] imx-drm display-subsystem: master bind failed: -517 With the approach provided in this post, it is possible to make this change manually, only by changing the flash.bin at U-boot for a non-m4 one. In order to make the changes to the flash.bin file, it’s needed to obtain the following files: - u-boot.bin from internal u-boot provided by NXP. - scfw_tcm.bin from SCFW porting kit - bl31.bin from ARM Trusted Firmware - SECO firmware container image Disclaimer The described procedures in this document target a GNU/Linux (Ubuntu 20.04 LTS) and it’s focused on iMX8QXP B0 + BSP L4.19.35_1.1.0. Required packages 1 - Install ARM64 ToolChain: 1.1 - Install ARM64 GCC and G++ cross-compilers: # apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu 2 - Install ARM32 GCC6 ToolChain: 2.1 - Download the ARM32 6 Toolchain and install it: $ mkdir ~/gcc_toolchain $ cp ~/Downloads/gcc-arm-none-eabi-6-2017-q2-update-linux.tar.bz2 ~/gcc_toolchain/ $ cd ~/gcc_toolchain/ $ tar xvjf gcc-arm-none-eabi-6-2017-q2-update-linux.tar.bz2 # apt-get update # apt-get install srecord 3 - Download MKimage 3.1 - Create a new directory desired to the packages: $ mkdir flash_build $ cp flash_build 3.1 - Clone the MKimage: $ git clone https://source.codeaurora.org/external/imx/imx-mkimage -b imx_4.19.35_1.1.0 4 - U-boot build 4.1 - Clone the U-boot  $ git clone https://source.codeaurora.org/external/imx/uboot-imx -b imx_v2019.04_4.19.35_1.1.0 $ cd uboot-imx 4.2 - Export the ARM64 ToolChain:  $ export ARCH=arm64 $ export CROSS_COMPILE=/usr/bin/aarch64-linux-gnu- 4.3 - Build it:  $ unset LDFLAGS $ make -j4 imx8qxp_mek_defconfig $ make 4.4 - Copy the binary files to the MKimage/iMX8QX directory:  $ cp spl/u-boot-spl.bin ../imx-mkimage/iMX8QX/ $ cp u-boot-nodtb.bin ../imx-mkimage/iMX8QX/ $ cd ..   5 - ARM Trusted Firmware 5.1 - Clone the imx-atf:  $ git clone https://source.codeaurora.org/external/imx/imx-atf -b imx_4.19.35_1.1.0 $ cd imx-atf 5.2 - Build it:  $ unset LDFLAGS $ make PLAT=imx8qx bl31 5.3 - Copy the binary files to the MKimage/iMX8QX directory:  $ cp build/imx8qx/release/bl31.bin ../imx-mkimage/iMX8QX/ $ cd ..   6 - SCFW 6.1 - Export the ARM32 GCC6 Toolchain:  $ export TOOLS=~/gcc_toolchain/ 6.2 - Download the BSP L4.19.35_1.1.0_SCFW and copy it to the flash_build directory:  $ cp ~/Downloads/imx-scfw-porting-kit-1.2.7.1.tar.gz $ tar xvzf imx-scfw-porting-kit-1.2.7.1.tar.gz $ cd packages/ $ chmod a+x imx-scfw-porting-kit-1.2.7.1.tar.gz $ ./imx-scfw-porting-kit-1.2.7.1.bin 6.3 - Build it to i.MX 8QXP MEK B0:  $ cd imx-scfw-porting-kit-1.2.7.1/src/ $ tar xvzf scfw_export_mx8qx_b0.tar.gz $ cd scfw_export_mx8qx_b0/ $ make qx R=B0 B=mek 6.4 - Copy the binary file to the MKimage/iMX8QX directory:  $ cp build_mx8qx_b0/scfw_tcm.bin ../../../../imx-mkimage/iMX8QX/ $ cp ../../../../ 7 - SECO Firmware Container Image 7.1 - Download the SECO firmware binaries and copy it to the flash_build directory $ cp ~/Downloads/firmware-imx-7.9.bin . $ chmod a+x firmware-imx-7.9.bin 7.2 - Copy the binary files to the MKimage/iMX8QX directory:  $ cp firmware-imx-7.9/firmware/seco/mx8qx-ahab-container.img /imx-mkimage/iMX8QX/ 8 - Build flash.bin 8.1 - In a new terminal, open the imx-mkimage directory: $ cd flash_build/imx-mkimage 8.2 - Build it:  $ make SOC=iMX8QX flash 8.3 - Deploy it to the SDCard:  $ sudo dd if=iMX8QX/flash.bin of=/dev/sdX bs=1k seek=32 && sync Now, you are able to use any non-rpmsg.dtb without kernel errors. Author: Pedro Jardim: pedro.jardim@nxp.com

Recently I published this i.MX Dev Blog post about the Gateworks plugin gst-variable-rtsp-server support for i.MX 6. Now, you can check how to use it on i.MX 8 SoCs as well. 1. Preparing the image In order to use gst-variable-rtsp-server plugin, prepare your machine and distro: Add the following line to conf/local.conf: IMAGE_INSTALL_append += "gstreamer1.0-rtsp-server gst-variable-rtsp-server" Download the attached patch and apply it by doing: $ cd <yocto_path>/sources/meta-fsl-bsp-release/ $ git am ~/Download/0001-Add-RTSP-support-for-i.MX-8-L4.14.78_ga1.0.0-or-olde.patch Note: This patch is not necessary for L4.14.98_ga2.0.0 BSP! Then, build the image with bitbake and deploy it to the SD card. 2. Video Test Source Example Server $ gst-variable-rtsp-server -p 9001 -u "videotestsrc ! v4l2h264enc ! rtph264pay name=pay0 pt=96" Client 2. Camera Example Server $ gst-variable-rtsp-server -p 9001 -u "v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! v4l2h264enc ! rtph264pay name=pay0 pt=96" Client In order to use VLC or other application as the client, just enter the URL as shown in the image below:

///////////////////////////create device node /dev/galcore///////////////////////////// $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/Kbuild MODULE_NAME ?= galcore /* define node name*/ $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/hal/os/linux/kernel/gc_hal_kernel_linux.h define DEVICE_NAME "galcore" $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/hal/os/linux/kernel/gc_hal_kernel_probe.c drv_init call ret = register_chrdev(major, DEVICE_NAME, &driver_fops); ///////////////////////////////opengles2 functios/////////////////////////////////////////// myandroid/device/fsl-proprietary/gpu-viv/lib/egl/libGLESv2_VIVANTE.so glActiveTexture glBindBuffer ... ... ... //those glxxxxxx call into sub_D40C int __fastcall sub_D40C(int a1, int a2, int a3) //address 0x0000D40C { int result; // r0@1 int v4; int v5; v4 = a2;   v5 = a3;   gcoOS_GetTLS(&v4);  //------------> goto libGAL.so   result = v4;   if ( v4 )     result = *(_DWORD *)(v4 + 36);   return result; } and $home/myandroid/device/fsl-proprietary/gpu-viv/lib/libGAL.so //export function signed int __fastcall gcoOS_GetTLS(void **a1) { ... ... gcoOS_GetTLS v4 = open("/dev/galcore", 2); ... ... } and device node /dev/galcore pass command into module galcore $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/hal/kernel/gc_hal_kernel.c gckKERNEL_Dispatch This document was generated from the following discussion: Share Vivante 3d gc2000 work flow

Check new updated version for with Morty here Step 1 : Get iMX Yocto AVS setup environment Review the steps under Chapter 3 of the i.MX_Yocto_Project_User'sGuide.pdf on the L4.X LINUX_DOCS to prepare your host machine. Including at least the following essential Yocto packages $ sudo apt-get install gawk wget git-core diffstat unzip texinfo \   gcc-multilib build-essential chrpath socat libsdl1.2-dev u-boot-tools Install the i.MX NXP AVS repo Create/Move to a directory where you want to install the AVS yocto build enviroment. Let's call this as <yocto_dir> $ cd <yocto_dir> $ repo init -u https://source.codeaurora.org/external/imxsupport/meta-avs-demos -b master -m imx7d-pico-avs-sdk_4.1.15-1.0.0.xml Download the AVS BSP build environment: $ repo sync Step 2: Setup yocto for Alexa_SDK image with AVS-SETUP-DEMO script: Run the avs-setup-demo script as follows to setup your environment for the imx7d-pico board: $ MACHINE=imx7d-pico DISTRO=fsl-imx-x11 source avs-setup-demo.sh -b <build_sdk> Where <build_sdk> is the name you will give to your build folder. After acepting the EULA the script will prompt if you want to enable: Sound Card selection The following Sound Cards are supported on the build: SGTL (In-board Audio Codec for PicoPi) 2-Mic Conexant The script will prompt if you are going to use the Conexant Card. If not then SGTL will be assumed as your selection Are you going to use Conexant Sound Card [Y/N]? Install Alexa SDK Next option is to select if you want to pre-install the AVS SDK software on the image. Do you want to build/include the AVS_SDK package on this image(Y/N)? If you select YES, then your image will contain the AVS SDK ready to use (after authentication). Note this AVS_SDK will not have WakeWord detection support, but it can be added on runtime. If your selection was NO, then you can always manually fetch and build the AVS_SDK on runtime. All the packages dependencies will be already there, so only fetching the AVS_SDK source code and building it is required. Finish avs-image configuration At the end you will see a text according with the configuration you select for your image build. Next is an example for a Preinstalled AVS_SDK with Conxant Sound Card support and WiFi/BT not enabled. ==========================================================   AVS configuration is now ready at conf/local.conf             - Sound Card = Conexant                                     - AVS_SDK pre-installed                                       You are ready to bitbake your AVS demo image now:               bitbake avs-image                                        ========================================================== Step 3: Build the AVS image Go to your <build_sdk> directory and start the build of the avs-image There are 2 options Regular Build: $ cd <yocto_dir>/<build_sdk> $ bitbake avs-image With QT5 support included: $ cd <yocto_dir>/<build_sdk> $ bitbake avs-image-qt5 The image with QT5 is useful if you want to add some GUI for example to render DisplayCards. Step 4 : Deploying the built images to SD/MMC card to boot on target board. After a build has succesfully completed, the created image resides at <build_sdk>/tmp/deploy/images/imx7d-pico/ In this directory, you will find the imx7d-pico-avs.sdcard image or imx7d-pico-avs-qt5.sdcard, depending on the build you chose on Step3. To Flash the .sdcard image into the eMMC device of your PicoPi board follow the next steps: Download the bootbomb flasher Follow the instruction on Section 4. Board Reflashing of the Quick Start Guide for AVS kit to setup your board on flashing mode. Copy the built SDCARD file $ sudo dd if=imx7d-pico-avs.sdcard of=/dev/sd bs=1M && sync $ sync Properly eject the pico-imx7d board: $ sudo eject /dev/sd NXP Documentation Refer to the Quick Start Quide for AVS SDK to fully setup your PicoPi board with Synaptics 2Mic and PicoPi i.mx7D For a more comprehensive understanding of Yocto, its features and setup; more image build and deployment options and customization, please take a look at the i.MX_Yocto_Project_User's_Guide.pdf document from the Linux documents bundle mentioned at the beginning of this document. For a more detailed description of the Linux BSP, u-boot use and configuration, please take a look at the i.MX_Linux_User's_Guide.pdf document from the Linux documents bundle mentioned at the beginning of this document.

OpenCV is a computer vision library originally developed by Intel. It is free for commercial and research use under the open source BSD license. The library is cross-platform. It focuses mainly on real-time image processing; as such, if it finds Intel's Integrated Performance Primitives on the system, it will use these commercial optimized routines to accelerate itself. Application OpenCV's application areas include: * 2D and 3D feature toolkits * Egomotion estimation * Face Recognition * Gesture Recognition * Human-Computer Interface (HCI) * Mobile robotics * Motion Understanding * Object Identification * Segmentation and Recognition * Stereopsis Stereo vision: depth perception from 2 cameras * Structure from motion (SFM) * Motion Tracking To support some of the above areas, OpenCV includes a statistical machine learning library that contains: * Boosting * Decision Trees * Expectation Maximization * k-nearest neighbor algorithm * Naive Bayes classifier * Artificial neural networks * Random forest * Support Vector Machine Installing OpenCV on i.MX 51 EVK Board running Ubuntu Linux Assuming that you already have the Ubuntu Linux running on your board, you can use this wiki page to guide you to get your USB camera running on your system in order to use real time image processing features of this library. In a brand new installation of Ubuntu some libraries is not installed by default, so you need to install them by your own hands (use synaptic to do that), here is the list of these libraries: libgtk2.0-dev libjpeg62-dev zlib1g-dev libpng12-dev libtiff4-dev libjasper-dev libgst-dev libgstreamer0.10-dev If you already have some of those libraries installed, make sure that is the DEV version. After installing those libraries you can download the stable OpenCV version here. Install it following the procedure below: 1 - untar the opencv package tar -xvzf opencv-1.1pre1.tar.gz  2 - change to OpenCV folder cd opencv-1.1.0  3 - configure the installation enabling gstreamer and letting to compile demo apps later ./configure --with-gstreamer --disable-apps You will get the following results: General configuration ================================================       Compiler:                         g++       CXXFLAGS:       DEF_CXXFLAGS:             -Wall -fno-rtti -pipe -O3 -fomit-frame-pointer       PY_CXXFLAGS:               -Wall -pipe -O3 -fomit-frame-pointer       OCT_CXXFLAGS:             -fno-strict-aliasing -Wall -Wno-uninitialized -pipe -O3 -fomit-frame-pointer        Install path:                      /usr/local  HighGUI configuration ================================================       Windowing system --------------       Use Carbon / Mac OS X:        no       Use gtk+ 2.x:                        yes       Use gthread:                         yes       Image I/O ---------------------       Use ImageIO / Mac OS X:       no       Use libjpeg:                            yes       Use zlib:                                yes       Use libpng:                             yes       Use libtiff:                               yes       Use libjasper:                          yes       Use libIlmImf:                          no             Video I/O ---------------------       Use QuickTime / Mac OS X:     no       Use xine:                                no       Use gstreamer:                        yes       Use ffmpeg:                             no       Use dc1394 & raw1394:     no       Use v4l:                                   yes       Use v4l2:                                 yes       Use unicap:                             no     Wrappers for other languages =========================================       SWIG Python                          no       Octave                                    no       Additional build settings ============================================       Build demo apps                      no Now run make ... 4 - Build OpenCV ./make 5 - Install OpenCV ./sudo make install if all steps above were executed properly, now you can compile the sample applications: 1 - change to samples/c directory cd samples/c 2 - change the build_all script mode to +x chmod +x build_all.sh 3 - run the script ./build_all.sh Now you can test. The results below were taken from the Laplacian filter sample processing in real-time images grabbed from an USB camera: Laplacian filter with USB Camera capture device Also, you can see how is it performance on a 3 windowed application performing color conversion and canny edge detection at the same time: http://www.youtube.com/watch?v=w9yQgdABT7c EOF !

BlueZ5 provides support for the core Bluetooth layers and protocols. It is flexible, efficient and uses a modular implementation. BlueZ5 has implemented the Bluetooth low level host stack for Bluetooth core specification 4.0 and 3.0+HS which includes GAP, L2CAP, RFCOMM, and SDP. Besides the host stack, BlueZ5 has also supported the following profiles itself or via a third party software. Profiles provided by BlueZ: A2DP 1.3 AVRCP 1.5 DI 1.3 HDP 1.0 HID 1.0 PAN 1.0 SPP 1.1 GATT (LE) profiles: PXP 1.0 HTP 1.0 HoG 1.0 TIP 1.0 CSCP 1.0 OBEX based profiles (by obexd): FTP 1.1 OPP 1.1 PBAP 1.1 MAP 1.0 Provided by the oFono project: HFP 1.6 (AG & HF)Supported Profiles BlueZ5 has been supported in the latest Freescale Linux BSP release, so it would be pretty easy to generate the binaries for Bluetooth core stack and its profiles. In order to support A2DP sink on a SabreSD board, the following software should be downloaded and installed onto the target rootfs too. sbc decoder version 1.3 (http://www.kernel.org/pub/linux/bluetooth/sbc-1.3.tar.gz) PulseAudio 5.0 (http://www.freedesktop.org/software/pulseaudio/releases/pulseaudio-5.0.tar.xz) PulseAudio package has some dependencies with bluetooth and sbc packages, and pulseaudio will detect if the two packages have been built and then decide which pulse plugin modules to be generated. So the building order will be 1) bluez5_utils or bluez_utils   2) sbc   3) pulseaudio. After compile and install the above software onto the target rootfs, you should be able to see the following executable under the directory /usr/bin From BlueZ5: bluetoothctl, hciconfig, hciattach (Needed by operating a UART bluetooth module) From PulseAudio: pulseaudio, pactl, paplay If the building dependency has been setup correctly, the following pulse plugin modules should be located under the directory /usr/lib/pulse-5.0/modules module-bluetooth-discover.so      module-bluetooth-policy.so        module-bluez5-device.so   module-bluez5-discover.so Edit the file /etc/dbus-1/system.d/pulseaudio-system.conf, and add the following lines in red: <policy user="pulse">     <allow own="org.pulseaudio.Server"/>    <allow send_destination="org.bluez"/>     <allow send_interface="org.freedesktop.DBus.ObjectManager"/> </policy> Edit the file /etc/dbus-1/system.d/bluetooth.conf, and add the following lines: <policy user="pulse">      <allow send_destination="org.bluez"/>      <allow send_interface="org.freedesktop.DBus.ObjectManager"/> </policy> Adding the following settings at the bottom of the pulseaudio system configuration file which locates in /etc/pulse/system.pa ### Automatically load driver modules for Bluetooth hardware .ifexists module-bluetooth-policy.so load-module module-bluetooth-policy .endif .ifexists module-bluetooth-discover.so load-module module-bluetooth-discover .endif load-module module-switch-on-connect load-module module-alsa-sink device_id=0 tsched=true tsched_buffer_size=1048576 tsched_buffer_watermark=262144 On the system that can automatically detect the alsa cards, the above line #13 should be removed.  Also make sure "auth-anonymous=1" is added to the following line, which can resolve the issue: "Denied access to client with invalid authorization data". load-module module-native-protocol-unix auth-anonymous=1 Selecting a audio re-sampling algorithm and configuring the audio output by adding the following settings to the file daemon.conf locating in /etc/pulse resample-method = trivial enable-remixing = no enable-lfe-remixing = no default-sample-format = s16le default-sample-rate = 48000 alternate-sample-rate = 24000 default-sample-channels = 2 Pulseaudio can be started as a daemon or as a system-wide instance. To run PulseAudio in system-wide mode, the program will automatically drop privileges from "root" and change to the "pulse" user and group. In this case, before launching the program, the "pulse" user and group needs to be created on the target system.  In the example below, "/var/run/pulse" is the home directory for "pulse" user. adduser -h /var/run/pulse pulse addgroup pulse-access adduser pulse pulse-access Because PulseAudio needs to access the sound devices, add the user "pulse" to the "audio" group too. adduser pulse audio Starting bluetoothd and pulseaudio: /usr/libexec/bluetooth/bluetoothd -d & pulseaudio --system --realtime & To verify if the pulseaudio has been set up correctly, you can play a local wave file by using the following command. If you can hear the sound, the system should have been configured correctly. paplay -vvv audio8k16S.wav After setting up the pulseaudio, launch bluetoothctl to pair and connect to a mobile phone. After connecting to a mobile phone, you should be able to see the following information in bluetoothctl console: [bluetooth]# show Controller 12:60:41:7F:03:00         Name: BlueZ 5.21         Alias: BlueZ 5.21         Class: 0x1c0000         Powered: yes         Discoverable: no         Pairable: yes         UUID: PnP Information           (00001200-0000-1000-8000-00805f9b34fb)         UUID: Generic Access Profile    (00001800-0000-1000-8000-00805f9b34fb)         UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb)         UUID: A/V Remote Control        (0000110e-0000-1000-8000-00805f9b34fb)         UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)         UUID: Message Notification Se.. (00001133-0000-1000-8000-00805f9b34fb)         UUID: Message Access Server     (00001132-0000-1000-8000-00805f9b34fb)         UUID: Phonebook Access Server   (0000112f-0000-1000-8000-00805f9b34fb)         UUID: IrMC Sync                 (00001104-0000-1000-8000-00805f9b34fb)         UUID: OBEX File Transfer        (00001106-0000-1000-8000-00805f9b34fb)         UUID: OBEX Object Push          (00001105-0000-1000-8000-00805f9b34fb)         UUID: Vendor specific           (00005005-0000-1000-8000-0002ee000001)         UUID: Audio Source              (0000110a-0000-1000-8000-00805f9b34fb)         UUID: Audio Sink                (0000110b-0000-1000-8000-00805f9b34fb)         Modalias: usb:v1D6Bp0246d0515         Discovering: no If you can see the audio sink UUID, you are ready to enjoy the bluetooth music now.

This document describes the i.MX 8MM EVK mini-SAS connectors features on Linux and Android use cases, covering the supported daughter cards, the process to change Device Tree (DTS) files or Boot images, and enable these different display options on the board.

This document describes all the i.MX 8 MIPI-CSI use cases, showing the available cameras and daughter cards supported by the boards, the compatible Device Trees (DTS) files, and how to enable these different camera options on the i.MX 8 boards. Plus, this document describes some Advanced camera use cases too, such as multiples cameras output using imxcompositor_g2d plugin, GStreamer zero-copy pipelines and V4L2 API extra-controls examples.

[中文翻译版] 见附件   原文链接： https://community.nxp.com/docs/DOC-342059 

[中文翻译版] 见附件   原文链接： https://community.nxp.com/docs/DOC-343528 

The following document contains a list of document, questions and discussions that are relevant in the community based on amount of views. If you are having a problem, doubt or getting started in i.MX processors, you should check the following links to see if your doubt is in there. Yocto Project Freescale Yocto Project main page‌ Yocto Training - HOME‌ i.MX Yocto Project: Frequently Asked Questions‌ Useful bitbake commands‌ Yocto Project Package Management - smart  How to add a new layer and a new recipe in Yocto  Setting up the Eclipse IDE for Yocto Application Development Guide to the .sdcard format  Yocto NFS &amp; TFTP boot  YOCTO project clean  Yocto with a package manager (ex: apt-get)  Yocto Setting the Default Ethernet address and disable DHCP on boot.  i.MX x Building QT for i.MX6  i.MX6/7 DDR Stress Test Tool V3.00  i.MX6DQSDL DDR3 Script Aid  Installing Ubuntu Rootfs on NXP i.MX6 boards  iMX6DQ MAX9286 MIPI CSI2 720P camera surround view solution for Linux BSP i.MX Design&amp;Tool Lists  Simple GPIO Example - quandry  i.MX6 GStreamer-imx Plugins - Tutorial &amp; Example Pipelines  Streaming USB Webcam over Network  Step-by-step: How to setup TI Wilink (WL18xx) with iMX6 Linux 3.10.53  Linux / Kernel Copying Files Between Windows and Linux using PuTTY  Building Linux Kernel  Patch to support uboot logo keep from uboot to kernel for NXP Linux and Android BSP (HDMI, LCD and LVDS)  load kernel from SD card in U-boot  Changing the Kernel configuration for i.MX6 SABRE  Android  The Android Booting process  What is inside the init.rc and what is it used for.  Others How to use qtmultimedia(QML) with Gstreamer 1.0

This document describes the i.MX 8QXP MEK mini-SAS connectors features on Linux and Android use cases, covering the supported daughter cards, the process to change Device Tree (DTS) files or Boot images, and enable these different display options on the board.

This document describes the i.MX 8MQ EVK HDMI output and mini-SAS connectors features on Linux and Android use cases, covering the supported daughter-board, the process to change Device Tree (DTS) files or Boot Images, and enable these different display options on the board.

meta-avs-demos Yocto layer meta-avs-demos is a Yocto meta layer (complementary to the NXP BSP release for i.MX) published on CodeAurora that includes the additional required packages to support  Amazon's Alexa Voice Services SDK (AVS_SDK) applications. The build procedure is the described on the README.md of the corresponding branch. We have 2 fuctional branches now: imx-alexa-sdk: Support for Morty based i.mx releases imx7d-pico-avs-sdk_4.1.15-1.0.0: legacy support for Jethro releases The master branch is only used to collect manifest files, that used with repo init/sync commands will fetch the whole environment for the 2 special supported boards: i.MX7D Pico Pi and i.MX8M EVK. However the meta-avs-demos can be used with any i.MX board either. Recipes to include Amazon's Alexa Voice Services in your applications. The meta-avs-demos provides the required recipes to build an i.MX image with the support for running Alexa SDK. The imx-alexa-sdk branch is based on Morty and kernel 4.9.X and it supports the next builds: i.MX7D Pico Pi i.MX8M EVK Generic i.MX board For the i.MX7D Pico Pi and i.MX8M EVK there is an extended support for additional (external) Sound Cards like: TechNexion VoiceHat: 2Mic Array board with DSPConcepts SW support Synaptics Card: 2 Mic with Sensory WakeWord support The Generic i.MX is for any other regular i.MX board supported on the official NXP BSP releases. Only the default soundcard (embedded) on the board is supported. Sensory wakeword is currently only enabled for those with ARMV7 architecture. To support any external board like the VoiceHat or Synaptics is up to the user to include the additional patches/changes required. Build Instructions Follow the corresponding README file to follow the steps to build an image with Alexa SDK support README-IMX7D-PICOPI.md README-IMX8M-EVK.md README-IMX-GENERIC.md

Description       this doc is explain how to develop a audio card driver base on i.MX6 platform. which explain the ASOC architecture struction basic knowledage and then give some sample for the audio driver development like: 1:NXP SGTL5000: NXP i.MX BSP sabrelite board default support it. 2: Wolfson WM8524.    A: 3.0.35 BSP support: i.MX6 setbox BSP support it:(which in elder fsl community link and out of data)    B: 3.14.28 BSP support pls check attachment: 3: Wolfson WM8960.     which include how to add the android middle-layer and driver, pls check attachment. 4: TI TLV320AIC3120      which include how to add the android middle-layer and driver, pls check attachment. 5: TI TLV320AIC3X   Products Product Category NXP Part Number URL MPU i.MX6 Family https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-6-processors:IMX6X_SERIES   Tools NXP Development Board URL i.MX6 SabreSDP https://www.nxp.com/design/development-boards:EVDEBRDSSYS#/collection=softwaretools&start=0&max=25&query=typeTax%3E%3Et633::archived%3E%3E0::Sub_Asset_Type%3E%3ETSP::deviceTax%3E%3Ec731_c380_c127_c126&sorting=Buy%2FSpecifications.desc&language=en&siblings=false which have a doc MX6X_ASOC_V5-20191115.pdf and related driver sample codes.

Overview As more and more communication required between online and offline, the QR code is widely used in the mobile payment, mobile small apps, industry things identification and etc. The i.MX6UL/ULL has the IP of CSI and PXP for camera connection and image CSC/FLIP/ROTATION acceleration. A LCDIF IP is supporting the display, but no 3D IP support. This means this low power and low end AP is very suitable for the industry HMI segment, which does not require a cool 3D graphic display, but a simple and straightforward GUI for interaction. QR code scanner is one of the use cases in the industry segment, which more and more customer are focusing on. The i.MX6UL CPU freq of i.MX6UL is about 500Mhz, and it does not have GPU IP, so a lightweight GUI and window system is required. Here we recommend the QT with wayland backend (without X11), which would make the window system small and faster than traditional X11 UI. Why chose QT is because of it has open source version, rich components, platform independent, good performance for embedded system and strong development staffs like QtCreator for creating application. How to enable the QT development environment, check this: Enable QT developement for i.MX6UL (v2)  Here I made a QR code scanner demo based on QT5.6 + QZXing (QR/Bar code scan engine) running on the i.MX6UL EVK board with a UVC camera (at least 640x480 resolution is required) and 480x272px LCD. Source code is open here (License Apache2.0): https://github.com/muddog/QRScanner  Implementation To do camera preview and capture, you must think on the gstreamer first, which is easy use and has the acceleration pads which implemented by NXP for i.MX6UL. Yes, it's very easy for you to enable the preview in console like: $ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=640,height=320 ! imxvideoconvert_pxp ! video/x-raw,format=RGB16 ! waylandsink It works under the i.MX6UL EVK, with PXP IP to do color space convert from YUY2 -> RGB16 acceleration, also the potential scaling of the image. The CPU loading of this is about 20-30%, but if you use the component of "videoconvert" to replace the "imxvideoconvert_pxp", we do CSC and scale by CPU, then the loading would increase to 50-60%. The "/dev/video1" is the device node for UVC camera, it may different in your environment. So our target is clear, create such pipeline (with PXP acceleration) in the QT application, and use a appsink to get preview images, do simple "sink" to one QWidget by drawing this image on the widget surface for preview (say every 50ms for 20fps). Then in other thread, we fetch the preview buffer in a fixed frequency (like every 0.5s), then feed it into the ZXing engine to decode the strings inside this image. Here are the class created inside the source code: ScannerQWidgetSink It act as a gstreamer sink for preview rendering. Init the pipeline, create a timer with timeout every 50ms. In the timer handler, we use appsink to copy the camera buffer from gstreamer, and tell the ViewfinderWidget to do update (re-draw event). ViewfinderWidget This class inherit from the QWidget, which draw the preview buffer as a QImage onto it's own surface by using QPainter. The QImage is created at the very begining with the image buffer created by the ScannerQWidgetSink. Because QImage itself does not maintain the image buffer, so the buffer must be alive during it's usage. So we keep this buffer during the ScannerQWidgetSink life cycle, copy the appsink buffer from pipeline to it for preview. MainWindow Create main window, which does not have title bar and border. Start any animation for the red line scan bar. Create instance of DecoderThread and ScannerQWidgetSink. Setup and start them. DecoderThread A infinite loop, to wait for a available buffer released by the ScannerQWidgetSink every 0.5s. Copy the buffer data to it's own buffer (imgData) to avoid any change to the buffer by sink when doing decoding. Then feed this copy of buffer into ZXing engine to get decoder result. Then show on the QLabel. Screenshot under wayland (weston) desktop: Customize Camera instance Now I use the UVC camera which pluged in the USB host, which device node is /dev/video1. If you want to use CSI or other device, please change the construction parameters for ScannerQWidgetSink(): sink = new ScannerQWidgetSink(ui->widget, QString("v4l2src device=/dev/video1")); Image resolution captured and review Change the static member value of ScannerQWidgetSink class: uint ScannerQWidgetSink::CAPTURE_HEIGHT = 480; uint ScannerQWidgetSink::CAPTURE_WIDTH = 640; Preview fps and decoding frequency Find the "framerate=20/1" strings in the ScannerQWidgetSink::GstPipelineInit(), change to your fps. You also have to change the renderTimer start timeout value in the ::StartRender(). The decoding frequency is determined by renderCnt, which determine after how many preview frames showed to feed the decoder. Main window size It's fixed size of main window, you have to change the mainwindow.ui. It's easy to do in the QtCreate Designer. FAQ Why not use CSI camera in demo? Honestly, I do not have CSI camera module, it's also DNP when you buying the board on NXP.com. So a widely used UVC camera is preferred, it's also easy for you to scan QR code on your phone, your display panel etc. Why not use QCamera to do preview and capture? The QCamera class in the Qtmultimedia component uses the camerabin2 gstreamer plugin, which create a very long pipeline for different usage of viewfinder, image capture and video encoder. Camerabin2 would eat too much CPU and memory resource, take picture and recording are very very slow. The preview of 30fps would eat about 70-80% CPU loading even I hacked it using imxvideoconvert_pxp instread of software videoconvert. Finally I give up to implement the QRScanner based on QCamera. How to make sure only one instance of QT app is running? We can use QSharedMemory to create a share memory with a unique KEY. When second instance of app is started, it would check if the share memory with this KEY is created or not. If the shm is there, it means there's already one instance running, it has to exit(). But as the QT mentioned, the QSharedMemory can not be destroyed correctly when app crashed, this means we have to handle each terminate signal, and do delete by ourselves: static QSharedMemory *gShm = NULL; static void terminate(int signum) {    if (gShm) {       delete gShm;       gShm = NULL;    }    qDebug() << "Terminate with signal:" << signum;    exit(128 + signum); } int main(int argc, char *argv[]) {    QApplication a(argc, argv);    // Handle any further termination signals to ensure the    // QSharedMemory block is deleted even if the process crashes    signal(SIGHUP, terminate ); // 1    signal(SIGINT, terminate ); // 2    signal(SIGQUIT, terminate ); // 3    signal(SIGILL, terminate ); // 4    signal(SIGABRT, terminate ); // 6    signal(SIGFPE, terminate ); // 8    signal(SIGBUS, terminate ); // 10    signal(SIGSEGV, terminate ); // 11    signal(SIGSYS, terminate ); // 12    signal(SIGPIPE, terminate ); // 13    signal(SIGALRM, terminate ); // 14    signal(SIGTERM, terminate ); // 15    signal(SIGXCPU, terminate ); // 24    signal(SIGXFSZ, terminate ); // 25    gShm = new QSharedMemory("QRScannerNXP");    if (!gShm->create(4, QSharedMemory::ReadWrite)) {       delete gShm;       qDebug() << "Only allow one instance of QRScanner";       exit(0);    } .....

MIPI can support video streaming over 1, 2, 3 and 4 lanes. On i.MX6 Sabre boards, the OV5640 camera supports 1 or 2 lanes and the NXP Linux Kernel uses 2 lanes as default. In order to use only one lane, follow the steps below: 1 - Change the board Device Tree on Linux Kernel. On file <linux kernel folder>/arch/arm/boot/dts/imx6qdl-sabresd.dtsi, find the entry "&mipi_csi" and change lanes from 2 to 1. 2 - Configure OV5640 to use only one lane instead of two. On file <linux kernel folder>/drivers/media/platform/mxc/capture/ov5640_mipi.c, change the register 0x300e value from 0x45 to 0x05. This register setup is located at struct ov5640_init_setting_30fps_VGA. 3 - Build the kernel and device tree files. 4 - Test the camera. Unit test can be used to test the video capture: /unit_tests/mxc_v4l2_overlay.out -di /dev/video1 -ow 1024 -oh 768 -m 1 5 - Checking if it's really using one lane. MIPI_CSI_PHY_STATE resgister (address 0x021D_C014) provides the status of all data and clock lanes. During video streaming using 2 lanes, the register value constantly changes its value between 0x0000_0300 and 0x0000_0330. When using only one lane, this register value constantly changes its value between 0x0000_0300 and 0x0000_0310. To read the register value during the stream, run the video test with &: /unit_tests/mxc_v4l2_overlay.out -di /dev/video1 -ow 1024 -oh 768 -m 1 & Now, run the memtool: /unit_tests/memtool -32 0x021dc014 1 i.MX6DL running mxc_v4l2_overlay.out with only one lane:

The document includes the following contents: (1)document how to port ov5646 to android jb4.2.2 (2) ov5645 driver for Linux 3.0.35 (3) ov5645 schematic based on i.MX6Q/DL (4)ov5645 for android camera HAL   [Note:]      P5V29A-0JG is a camera module based on OV5645, and PAO532-0JG is based on OV5640, both manufactured by NINGBO SUNNY OPOTECH CO.LTD (China), If customer wants to use them on i.MX6 platform, can send me email to ask for datasheets of P5V29A & PAO532 , or discuss corresponding questions on porting.   Email: weidong.sun@freescale.com