Information about the transition from the NXP Demo Experience to GoPoint for i.MX Application Processors.

On this tutorial we will review the implementation of Flutter on the i.MX8MP using the Linux Desktop Image. Please find more information about Flutter using the following link: Flutter: Option to create GUIs for Embedded System... - NXP Community Requirements: Evaluation Kit for the i.MX 8M Plus Applications Processor. (i.MX 8M Plus Evaluation Kit | NXP Semiconductors) NXP Desktop Image for i.MX 8M Plus (GitHub - nxp-imx/meta-nxp-desktop at lf-6.1.1-1.0.0-langdale) Note: This tutorial is based on the NXP Desktop Image Ubuntu 22.04 with Yocto version 6.1.1 – Langdale. Steps: 1. First, run commands to update packages. $ sudo apt update $ sudo apt upgrade 2. Install Flutter for Linux using the following command. $ sudo snap install flutter --classic 3. Run the command to verify the correct installation. $ flutter doctor With this command you will find information about the installation. The important part for our purpose is the parameter "Linux toolchain - develop for Linux desktop". 4. Run the command “flutter create .” to create a flutter project, this framework will create different folders and files used to develop the application.  $ cd Documents $ mkdir flutter_hello $ cd flutter_hello $ flutter create .​ 5. Finally, you can run the “hello world” application using: $ flutter run Verify the program behavior incrementing the number displayed on the window.  

One of the most popular use cases for embedded systems are projects destinated to show information and interact with users. These views are called GUI or Graphic User Interface which are designed to be intuitive, attractive, consistent, and clear. There are many tools that we can use to achieve great GUIs, mostly implemented for platforms such as Web, Android, and iOS. Here, we will need to introduce the concept of framework, basically, it is a set of tools and rules that provides a minimal structure to start with your development. Frameworks usually comes with configuration files, code snippets, files and folders organization helping us to save time and effort. Also, it is important to review the concept of SDK or Software Development Kit which is a set of tools that allows to build software for specific platforms. Usually supplies debugging tools, documentation, libraries, API’s, emulators, and sample code. Flutter is an open-source UI software development kit by Google that help us to create applications with great GUIs on different platforms from a single codebase. Depends on the reference, you can find Flutter defined as a framework or SDK and both are correct, however, an SDK could be a best definition thanks to Flutter supplies a wide and complete package to create an application in which framework is also included. This article is aimed at those that are in a prototyping stage looking for a different tool to develop projects. Also, this article pretends to be a theoretical introduction explaining the most important concepts. However, is a good practice to learn more about reviewing the official documentation from Flutter. (Flutter documentation | Flutter) Here is the structure used throughout this article: What is Flutter? Flutter details Platforms Programming language Official documentation Flutter for embedded systems What is Flutter? Flutter was officially released by Google in December 2018 with a main aim, to give developers a tool to create applications natively compiled for mobile (Android, iOS), web and desktop (Windows, Linux) from a single codebase. It means that as a developer, Flutter will create a structure with minimal code, configuration files, build files for each operating system, manifests, etc. in which we will add our custom code and finally build this code for our preferred OS. For example, we can create an application to review fruit and vegetable information and compile for Android and iOS with the same code. A basic Flutter development process based on my experience looks like the following diagram: Flutter has the following key features: Cross-platform development. Flutter allows the developer to create applications for different platforms using a single codebase. It means that you will not need to recreate the application for each platform you want to support.   Hot-reload. This feature allows the developer to see changes in real time without restarting the whole application, this results in time savings for your project.   High Performance Flutter apps achieve high performance due to the app code is compiled to native ARM code. With this tool no interpreters are involved.   UI Widgets Flutter supplies a set of widgets (UI components such as boxes, inputs text, buttons, etc.) predefined by UI systems guidelines Material on Android and Cupertino for iOS. Source: Material 3 Design Kit | Figma Community Source: Design - Apple Developer   Great community support. This feature could be subjective but, it is useful when we are developing our project find solutions to known issues or report new ones. Because of Flutter is an open source and is widely implemented in the industry this tool owns a big community, with events, forums, and documentation. Flutter Details Supported Platforms With Flutter you can create applications for: Android iOS Linux Debian Linux Ubuntu macOS web Chrome, Firefox, Safari, Edge Windows Supported deployment platforms | Flutter Programming Language Flutter use Dart, a programming language is an open-source language supported by Google optimized to use on the creation of user interfaces. Dart key features: Statically typed. This feature helps catching errors making the code robust ensuring that the variable’s value always match with the declared variable’s type. Null safety. All variables on Dart are non-nullable which means that every variable must have a non-null value avoiding errors at execution time. This feature also, make the code robust and secure. Async/Await. Dart is client-optimized which means that this language was specially created to ensure the best performance as a client application. Async/Await is a feature part of this optimization making easier to manage network requests and other asynchronous operations. Object oriented. Dart is an object-oriented language with classes and mixin. This is especially useful to use on Flutter with the usage of widgets. Compiler support of Just-In-Time (JIT) and Ahead-of-Time (AOT) JIT provides the support that enables the Hot Reload Flutter feature that I mentioned before. It is a complex mechanism, but Dart “detects” changes in your code and execute only these changes avoiding recompiling all the code. AOT compiler produces efficient ARM code improving start up time and performance. Official documentation Flutter has a rich community and documentation that goes from UI guidelines to an Architectural Overview. You can find the official documentation at the following links: Flutter Official Documentation: Flutter documentation | Flutter Flutter Community: Community (flutter.dev) Dart Official Documentation: Dart documentation | Dart Flutter for embedded systems So far, we know all the excellent features and platforms that Flutter can support. But, what about the embedded systems? On the official documentation we can find that Flutter may be used for embedded systems but in fact there is no an official supported platform. This SDK has been supported by their community, specially there is one repository on GitHub supported by Sony that provides documentation and Yocto recipes to support Flutter on embedded Linux. To understand the reason to differentiate between Flutter for Linux Desktop with official support and to create a specific Flutter support for embedded Linux is important to describe the basics of Flutter architecture. Based on the Flutter documentation the system is designed using layers that can be illustrated as follows:   Source: Flutter architectural overview | Flutter We can see as a top level “Framework” which is a high-level layer that includes widgets, tools and libraries that are in contact with developers. Below “Framework,” the layer “Engine” is responsible of drawing the widgets specified in the previous layer and provides the connection between high-level and low-level code. This layer is mostly written in C++ for this reason Flutter can achieve high performance running applications. Specifically for graphics rendering Flutter implements Impeller for iOS and Skia for the rest of platforms. The bottom layer is “Embedder” which is specific for each target and operating system this layer allows Flutter application to run as a native app providing the access to interact with different services managed by the operating systems such as input, rendering surfaces and accessibility. This layer for Linux Desktop uses GTK/GDK and X11 as backend that is highly dependent of unnecessary libraries and expensive for embedded systems which have constrained resources for computation and memory. The work around founded by Sony’s Flutter for Embedded Linux repository is to change this backend using a widely implemented backend for embedded systems Wayland. The following image illustrates the difference between Flutter for Linux Desktop and Flutter for Embedded Linux.   Source: What's the difference between Linux desktop and Embedded Linux · sony/flutter-embedded-linux Wiki · GitHub   Source: What's the difference between Linux desktop and Embedded Linux · sony/flutter-embedded-linux Wiki · GitHub Here is the link to the mentioned repository: GitHub - sony/flutter-elinux: Flutter tools for embedded Linux (eLinux) Finally, I would like to encourage you to read the official Flutter documentation and consider this tool as a great option compared to widely used tools on embedded devices such as Qt or Chromium. Also, please have a look to a great article written by Payam Zahedi delving into the implementation of Flutter for Embedded Linux measuring performance and giving conclusions about the usage of Flutter in embedded systems. (Flutter on Embedded Devices. Learn how to run Flutter on embedded… | by Payam Zahedi | Snapp Embedded | Medium).    

GUI Guider version: 1.6.0 LVGL version: v8.3.5 Host software requirements: Ubuntu 20.04, Ubuntu 22.04 or Debian 12 Hardware requirements: Evaluation Kit for the i.MX 93 Applications Processor. (i.MX 93 Evaluation Kit | NXP Semiconductors) On this guide we will use the IMX-MIPI-HDMI accessory board to connect the iMX93 with a HDMI Monitor. (IMX-MIPI-HDMI Product Information|NXP) This board is usually provided with the iMX8M Mini and the iMX8M Nano.  Steps: 1. Copy your project from the folder GUI-Guider-Projects to your Linux PC.  2. Build an image for iMX93 using The Yocto Project.    a. Based on iMX Yocto Porject Users Guide set directories and download the repo $ mkdir imx-bsp-6.1.1-1.0.0 $ cd imx-bsp-6.1.1-1.0.0 $ repo init -u https://github.com/nxp-imx/imx-manifest -b imx-linux-langdale -m imx-6.1.1-1.0.0.xml $ repo sync Use distro fsl-imx-xwayland and select machine imx93evk and use this commnad with a build folder name: $ MACHINE=imx93evk DISTRO=fsl-imx-xwayland source ./imx-setup-release.sh - b bld-imx93evk b. Use bitbake command to start the build process. Also, add the -c populate_sdk to get the toolchain. $ bitbake imx-image-multimedia -c populate_sdk  c. Install the Yocto toolchain located on <build-folder>/tmp/deploy/sdk/.  $ sudo sh ./fsl-imx-xwayland-glibc-x86_64-imx-image-multimedia-armv8a-imx93evk-toolchain-6.1-langdale.sh d. Install ninja utility on the build host $ sudo apt install ninja-build e. For Ubuntu 20.04 and Ubuntu 22.04, copy the lv_conf.h file from lvgl-simulator to lvgl $ cp lvgl-simulator/lv_conf.h lvgl/ f. Change the interpreter on build.sh from #!/bin/sh to #!/bin/bash. This is an important step! g. Then, enter to linux folder and use the following commands to make build.sh executable $ dos2unix build.sh $ chmod +x build.sh h. Execute the build.sh $ ./build.sh i. Copy the binary to the iMX93 using a USB or SCP.  2. On the target iMX93 follow these steps. a. On Uboot, use fatls interface device:partition fatls mmc 0:1 (Device 0 : Partition 1) With this command, we will be able to list device tree files. => fatls mmc 0:1 b. Select imx93-11x11-evk-rm67199.dtb and use the command editenv fdtfile  => editenv fdtfile Output example edit: imx93-11x11-evk-rm67199.dtb c. In edit command line put the selected device tree .dtb d. Use saveenv command to save environment and continue with the boot process. e. Finally, run the GUI Application $ ./gui_guider&   I hope this article will be helpful. Best regards, Brian.

vpuwraper can fulfill VPU decoder/encoder, if customer’s user case is simple, for example they just need to encode yuv stream to H264, or decode H264 stream to yuv, There is no need to use gstreamer or V4L2 complex framework, you can use vpuwraper. Platform: i.MX8MP + L5.4.70.2.3.0 Build Procedure: mkdir vpu cd vpu git clone https://github.com/nxp-imx/imx-vpuwrap   cd imx-vpuwrap/ git tag -l   git switch -c rel_imx_5.4.70_2.3.0   source ../../.././5.4.70.2.3.0/sdk/environment-setup-aarch64-poky-linux   make -f Makefile_8mp   Test on i.MX8MP EVK board Pls find attached test log for decode and encode If busChromaU in YUV file is null, you will failed to encode it,pls apply patch vpuwraper patch for L5.4.70.2.3.0.patch to fix t If YUV file is interleave format, you need to add add interleave parameter : -interleave 1 ./test_enc_arm_elinux -i test.yuv -o aaa.h264 -f 2 -w 176 -h 96 -interleave 1   Thanks, Lambert

In this article, I will explain how to set up the iMX8M Plus to use the 4K Dart BCON Basler Camera module. Requirements: Evaluation Kit for the i.MX 8M Plus Applications Processor. (i.MX 8M Plus Evaluation Kit | NXP Semiconductors) Basler Camera for i.MX 8M Plus (4K dart BCON for MIPI camera module for i.MX 8M Plus | NXP Semiconductors). Embedded Linux for i.MX Applications Processors (Embedded Linux for i.MX Applications Processors | NXP Semiconductors) (For this example we will use BSP version Linux 5.15.71_2.2.0) Serial Console Emulator Basler Camera Specifications and Manuals: Basler Camera Specifications at this link: Embedded Vision Kits daA3840-30mc-IMX8MP-EVK - Embedded Vision Kits (baslerweb.com). Basler Manual to identify and setting up the hardware at this link: daA3840-30mc-IMX8MP-EVK | Basler Product Documentation (baslerweb.com) Basler Camera Module out-of-box with i.MX 8M Plus Applications Processor. (Video: Basler Camera Module out-of-box with i.MX 8M Plus Applications Processor | NXP Semiconductors) Steps After setting up the hardware we will need to turn on the iMX8M Plus and follow these steps: 1. Stop the boot process on Uboot by pressing any key. 2. Use the following command to list interfaces. => mmc list Output example => FSL_SDHC: 1 (SD) => FSL_SDHC: 2 The above command will show you the device number in this example for SD, the device number is 1. 3. Then use fatls <interface> <device[:partition]> [<directory>] fatls mmc 1:1 (Device 1 : Partition 1) With this command, we will be able to list device tree files. => fatls mmc 1:1 4. Select imx8mp-evk-basler.dtb or imx8mp-evk-dual-basler.dtb and use the command editenv fdtfile.  => editenv fdtfile Output example edit: imx8mp-evk-basler.dtb 5. In edit command line put the selected device tree (*.dtb). 6. Use saveenv command to save environment and continue with the boot process. 7. Using the terminal and go to /opt/imx8-isp/bin and execute the script run.sh. $ ./run.sh -c basler_1080p60 -lm 8. Use the command gst-device-monitor-1.0 to list devices. Here you will find the path to the camera device. $ gst-device-monitor-1.0 Output example Device found: name : VIV class : Video/Source caps : video/x-raw, format=YUY2, width=[ 176, 4096, 16 ], height=[ 144, 3072, 8 ], pixel-aspect-ratio=1/1, framerate={ (fraction)30/1, (fraction)29/1, (fraction)28/1, (fraction)27/1, (fraction)26/1, (fraction)25/1, (fraction)24/1, (fraction)23/1, (fraction)22/1, (fraction)21/1, (fraction)20/1, (fraction)19/1, (fraction)18/1, (fraction)17/1, (fraction)16/1, (fraction)15/1, (fraction)14/1, (fraction)13/1, (fraction)12/1, (fraction)11/1, (fraction)10/1, (fraction)9/1, (fraction)8/1, (fraction)7/1, (fraction)6/1, (fraction)5/1, (fraction)4/1, (fraction)3/1, (fraction)2/1, (fraction)1/1 } ... properties: udev-probed = true device.bus_path = platform-vvcam-video.0 sysfs.path = /sys/devices/platform/vvcam-video.0/video4linux/video2 device.subsystem = video4linux device.product.name = VIV device.capabilities = :capture: device.api = v4l2 device.path = /dev/video2 v4l2.device.driver = viv_v4l2_device v4l2.device.card = VIV v4l2.device.bus_info = platform:viv0 v4l2.device.version = 393473 (0x00060101) v4l2.device.capabilities = 2216693761 (0x84201001) v4l2.device.device_caps = 69206017 (0x04200001) gst-launch-1.0 v4l2src device=/dev/video2 ! ... 9. Finally, use gstreamer to verify proper operation. (With this gstreamer pipeline you will see a new window with the camera output. Then, just rotate the lens to acquire the correct focus) $ gst-launch-1.0 -v v4l2src device=/dev/video2 ! "video/x-raw,format=YUY2,width=1920,height=1080" ! queue ! imxvideoconvert_g2d ! waylandsink Basic description of Gstreamer Pipeline gst-launch-1.0 -v: The option -v enables the verbose mode to get detailed information of process. v4l2src device=/dev/video2: Select input device in this case the camera is on path /dev/video3. "video/x-raw,format=YUY2,width=1920,height=1080": Received format from camera. queue: This command is a buffer between camera recording process and the following image process, this command help us to interface two process and prevent blocking where each process has different speeds, in other words, when a process A is faster than process B. imxvideoconvert_g2d: This proprietary plugin uses hardware acceleration to perform rotation, scaling, and color space conversion on video frames. waylandsink : This command creates its own window and renders the decoded frames processed previously. 10. Result     I hope this article will be helpful. Best regards, Brian.

Hello everyone, We have recently migrated our Source code from CAF (Codeaurora) to Github, so i.MX NXP old recipes/manifest that point to Codeaurora eventually will be modified so it points correctly to Github to avoid any issues while fetching using Yocto. Also, all repo init commands for old releases should be changed from: $ repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b <branch name> [ -m <release manifest>] To: $ repo init -u https://github.com/nxp-imx/imx-manifest -b <branch name> [ -m <release manifest>] This will also apply to all source code that was stored in Codeaurora, the new repository for all i.MX NXP source code is: https://github.com/nxp-imx For any issues regarding this, please create a community thread and/or a support ticket. Regards, Aldo.

i.MX8 series contains internal HiFi4 DSP. It is targeted for Audio related signal processing. SOF (Sound Open Firmware) is open source audio DSP firmware, driver and SDK. This document introduces basic theory about IIR/FIR digital filters, how to design IIR/FIR digital filters and the Equalizer filters implementation by SOF. After that, the document also describes how HiFi4 DSP MAC engine accelerate the EQ filters calculation.

Hello, here Jorge. On this post I will explain how to configure, record and play audio using an i.MX 8MIC-RPI-MX8 Board. Requirements: I.MX 8M Mini EVK Linux Binary Demo Files - i.MX 8MMini EVK (L5.15.52_2.1.0) i.MX 8MIC-RPI-MX8 Board Serial console emulator (Tera Term, Putty, etc.) Headphones/speakers The 8MIC-RPI-MX8 accessory board is designed for voice enabled application prototyping and development on the i.MX 8M family. The board plugs directly into the 40-pin expansion connector on the i.MX 8M Mini and Nano EVK’s. Some features about this board are: 8 PDM Microphones 8 monochrome LEDs 4 multi-color LEDs 2 status LEDs 4 pushbuttons Microphone Mute Switch Microphone geometry switch Connecting the i.MX 8MIC-RPI-MX8 Board. The i.MX 8MIC-RPI-MX8 Board has a 40-pin expansion connector that you can plug it directly to the EVK board. Ensure that pin 1 of the 8MIC-RPI-MX8 is aligned with pin 1 on the EVK J1001 as is showed on the next figure:  Selecting the device tree on the board. Once the pre-compiled image is flashed on the board (Flashing Linux BSP using UUU) and you connected the 8MIC-RPI-MX8 it is necessary to select the correct device tree to handle 8MIC board. On U-boot check the available .dtb files on the BSP using the next command: u-boot=> fatls mmc 2:1 And you will get the corresponding list of .dbt files:  On this case we are working with an I.MX 8M Mini EVK and the corresponding .dtb file is: imx8mm-evk-8mic-revE.dtb To select it you need to set the environment variable and save it with: u-boot=> setenv fdtfile imx8mm-evk-8mic-revE.dtb u-boot=> saveenv Doble check it using: u-boot=> printenv fdtfile   Now it is time to boot Linux using the next command: u-boot=> boot Recording audio with the i.MX 8MIC-RPI-MX8 Board. The Advanced Linux Sound Architecture (ALSA) provides audio and MIDI functionality to the Linux operating system. ALSA has the following significant features: Efficient support for all types of audio interfaces, from consumer sound cards to professional multichannel audio interfaces. Fully modularized sound drivers. SMP and thread-safe design. User space library (alsa-lib) to simplify application programming and provide higher level functionality. Support for the older Open Sound System (OSS) API, providing binary compatibility for most OSS programs. Once we are on Linux, we can check our audio codecs detected on the board using: arecord -l   Now, to record audio we need to use the ALSA arecord command to start recording with IMX8 boards, there are different options that you can check on the next link. On this case we are going to use the next: arecord -D hw:imxaudiomicfil -c8 -f s16_le -r48000 -d10 sample.wav -D: selects the device. -c: selects the number of channels on the recording. -f: selects the format. -r: selects the sample rate. -d: determinate the duration recording time in seconds. sample.wav: Is the name of the resulting audio file. Running the last command, we started to record audio. It is time to make some noise and record it!   Playing audio from IMX8 boards. Now it is time to connect our headphones or speakers to the jack.   Also, as on arecord command you can check the devices where you can play audio from the board using the next command: aplay -l And you will get all the codecs to play audio:   To play our recordings we need to use the ALSA aplay command, it is important to select the correct audio codec to hear the audio from the jack on the board: aplay -Dplughw:3,0 sample.wav -D: selects the device. sample.wav: Is the name of audio file to play   Hope this will helpful for people who wants to record audio using PDM microphones and playing audio from IMX8 boards. Best regards.

i.MX8 VPU hardware decoder support below video codec: H.265 HEVC Main Profile 4Kp60 Level 5.1 H.264 AVC Constrained Baseline, Main and High profile H.264 MVC WMV9 / VC-1 Simple, Main and Advanced Profile MPEG 1 and 2 Main Profile at High Level AVS Jizhun Profile (JP) MJPEG4.2 ASP, H.263, Sorenson Spark Divx 3.11, with Global Motion Compensation (GMC) ON2/Google VP6/VP8 RealVideo 8/9/10 JPEG and MJPEG A/B Baseline   i.MX8 VPU Linux driver is implemented based on V4L2 standard. Chromium beside software video decoding, it also support hardware video decoder(VideoDecodeAccelerator),  there are some kind of VideoDecodeAccelerator, one of them is V4L2VDA. Please note V4L2VDA is using V4l2 api, so it is possible that change V4L2VDA to enable Chromium hardware video playback on i.MX8.   This doc share patch to add chromium video decode accelerate by using i.MX8QM/i.MX8QXP VPU. It will support chromium H.264, H.265, VP8 hardware video decode. H.264 and H.265 need use mp4 container. VP8 use webm container.   HW: i.MX8QM/i.MX8QXP MEK board, 1080P HDMI display, mouse, keyboard SW: i.MX8 5.10.72_2.2.2 yocto bsp release(which included chromium 91.0), and patch in this doc   Patch description: imx8-5.10.72-vpudrv-update.diff, update i.MX8  5.10.72_2.2.2 kernel vpu driver to https://source.codeaurora.org/external/imx/linux-imx/commit/drivers/mxc/vpu_malone?h=lf-5.15.y&id=fa7c67e2c9ed4fb8392fa258f931d6996339a17a chromium-ozone-wayland_91.0.4472.114.bb.diff, change meta-browser/meta-chromium/recipes-browser/chromium/chromium-ozone-wayland_91.0.4472.114.bb for adding some compile flags, etc. 5.10.72-merge.patch, this patch change chromium source code to add video decode accelerate by using i.MX8 VPU.   Build steps: 1>Download i.MX8 5.10.72_2.2.2 yocto release from nxp.com 2>apply chromium-ozone-wayland_91.0.4472.114.bb.diff to change meta-browser/meta-chromium/recipes-browser/chromium/chromium-ozone-wayland_91.0.4472.114.bb 3>put 5.10.72-merge.patch to folder path_of_yocto-5.10.72-2.2.2/sources/meta-browser/meta-chromium/recipes-browser/chromium/files/ 3>apply imx8-5.10.72-vpudrv-update.diff to i.MX8 5.10.72_2.2.2 kernel 4>under the yocto image build folder, add "CORE_IMAGE_EXTRA_INSTALL += "chromium-ozone-wayland" to file path_of_yocto-5.10.72-2.2.2/folder-of-bld/conf/local.conf 5>run bitbake to build rootfs image   Test steps: After system boot up, put some video clip under /home/root/video then run below cmd (do not run chromium without any parameter, as that will start chromium with some other setting, you can check /usr/lib/chromium/chromium-wrapper) "/usr/lib/chromium/chromium-bin   --no-sandbox --ozone-platform=wayland --enable-features=VaapiVideoDecoder  --enable-accelerated-video-decode   --enable-clear-hevc-for-testing --ignore-gpu-blacklist --window-size=1920,1180  /home/root/video" then use mouse to click video clip and will start playback.   Reference: https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-processors:IMX8-SERIES https://www.nxp.com/design/software/embedded-software/i-mx-software/embedded-linux-for-i-mx-applications-processors:IMXLINUX https://www.chromium.org/audio-video/#:~:text=codec%20and%20container%20support https://github.com/igel-oss/meta-browser-hwdecode/blob/master/recipes-chromium/chromium/files/0001-Add-support-for-V4L2VDA-on-Linux.patch      

On behalf of Gopise Yuan. A collection of several GST debugging tips and known-how. When you need to play onto a DRM layer/plane directly without going through compositor, kmssink should be a good choice: // kmssink, with scale and adjust alpha property (opaque) and zpos (this requires kmssink>=1.16): gst-launch-1.0 filesrc location=/media/AVC-AAC-720P-3M_Alan.mov ! decodebin ! imxvideoconvert_g2d ! kmssink plane-id=37 render-rectangle="<100,100,720,480>" can-scale=false plane-properties=s,alpha=65535,zpos=2 When using playbin, you can still customize the pipeline besides the sink plugin, e.g. add a converter plugin: // Playbin with additional customization on converter before sink: gst-launch-1.0 playbin uri=file:///mnt/MP4_H264_AAC_1920x1080.mp4 video-sink="imxvideoconvert_g2d ! video/x-raw,format=BGRA,width=1920,height=1080 ! kmssink plane-id=44" GST can generate a pipeline graph for analyzing the pipeline in a intuitive manner: // Generate pipeline graph: 1. Export GST_DEBUG_DUMP_DOT_DIR=<dump-folder>, GST_DEBUG=4 2. Run pipeline with gst-launch or others. 3. Copy all dump files (.dot) from <dump-folder>. Note: one dump file will be created for each state transaction. Normally, what we need will be PAUSE_READY or READY_PAUSE, after which pipeline has been setup. 4. Convert the .dot file to PDF with Graphviz: dot -Tpdf 0.00.03.685443250-gst-launch.PAUSED_READY.dot > pipeline_PAUSED_READY.pdf  

1. Intro   This document contains instructions to run run the SAI low power audio demo on the i.MX 8M Plus EVK. Here, the  RPSMG to allows audio to be passed from the A53 cluster running Linux to the M7 core. The latter controls the on board WM8960 audio codec,  which is connected to a 3.5 mm audio jack that allow us to play music using headphones. I will show the necessary steps to make the demo work and will add some GStreamer examples to demonstrate the demo's capabilities.   TBD: update this with a nice diagram that depicts the A53 and M7 RPMSG channel. 2. Requirements   Hardware  MX 8M Plus EVK Headphones with 3.5 mm audio jack Type-C power supply for i.MX 8M Plus EVK Micro USB to USB adapter cable Software  A recent prebuilt Linux BSP image from NXP.com ( we tested this on 5.15.35 and 5.15.5 releases) Windows 10 or Ubuntu 20.04 Workstation MCUXpresso SDK for i.MX 8M Plus ( available from:  Welcome | MCUXpresso SDK Builder (nxp.com)) 3. Reference documentation for this example   MCUXpresso SDK   [1] Getting Started with MCUXpresso SDK for EVK-MIMX8MP     Available within the MCUXpresso SDK package:  \{INSTALL PATH}\SDK_X_X_X_EVK-MIMX8MP\docs    [2] SAI low power audio README file Contains instructions for the SAI Low Power Audio Demo.  Available within the MCUXpresso SDK package: \{INSTALL PATH}\SDK_X_X_X_EVK-MIMX8MP\boards\evkmimx8mp\demo_apps\sai_low_power_audio   4. Downloading a pre-built Linux BSP image for the i.MX 8M Plus   I will make use of the prebuilt Linux Image for the i.MX 8M Plus EVK for demonstrating the demo works.  At the moment of writing this time, I used the 5.15.32 release, although there are older releases like 5.10.5 that I tested and proved to work with no issues. This SAI Low Power Audio Demo shall work for other processors on the i.MX 8M family. Although specific instructions ( e.g. load address for M-core binary load) might require some adaptation. For M-core load address, please refer to the specific MCUXpresso SDK documentation for each processor. The prebuilt Linux image (5.15.32) for the i.MX 8M Plus EVK can be downloaded from here: https://www.nxp.com/webapp/Download?colCode=L5.15.32_2.0.0_MX8MP&appType=license You can download other releases from here: Embedded Linux for i.MX Applications Processors | NXP Semiconductors . Select a version and a board and select download. 5. Flashing the BSP image   If you are using an Ubuntu 20.04 workstation, I recommend you to flash the image using dd. For this, you can refer to the i.MX Linux User's Guide: Section - 4.3.2 Copying the full SD card image - https://www.nxp.com/docs/en/user-guide/IMX_LINUX_USERS_GUIDE.pdf sudo dd if=.wic of=/dev/sdx bs=1M && sync NOTE: when using dd, ALWAYS, double check the of device that you are about to writing. Messing up with another location or partition will harm your system   If you are following this document on a Windows machine: You can use the Universal Update Utility (UUU) to flash your image on either the board's eMMC or SD card. Document named UUU.pdf shall serve as your reference guide for further instructions and flashing examples. It is available along with UUU binary here: https://github.com/NXPmicro/mfgtools/releases Two examples are shown below for your convenience:                                     SD card flash                                                 uuu -b sd_all bootloader rootfs.sdcard.bz2                                     eMMC flash                                                 uuu -b emmc_all bootloader rootfs.sdcard.bz2        uuu uuu.auto NOTE: UUU is also compatible with Ubuntu NOTE: there are other engineers who like to use BalenaEtcher for flashing their BSP images. I have tested it and works on both Ubuntu and Windows 10 machines.   6. Preparing the BSP and booting up M7 core  using U-Boot   I am writing this upon the instructions contained on the README file for the low power audio example  [2]. Instructions ready to copy and paste will follow:   Instruct U-Boot to pass to the kernel the rpmsg device tree to enable communication between the A53 cluster and the M7 one: u-boot=>setenv fdtfile imx8mp-evk-rpmsg.dtb u-boot=>saveenv Load the M7 example: u-boot=>setenv mmcargs 'setenv bootargs ${jh_clk} console=${console} root=${mmcroot} snd_pcm.max_alloc_per_card=134217728' u-boot=>saveenv Now, we need to load the M4 with the demo. Refer to [1] for further information. If running the BSP on an SD card, make sure the example binary is listed on the boot partition as follows: fatls mmc 1:1 You shall see something similar to this:             imx8mp_m7_TCM_sai_low_power_audio.bin Open the serial terminal emulator for the M7. Out of the fourth ports listed when we plug the i.MX 8M Plus serial debug cable to the PC, the M7 is typically the last one listed.   All the serial ports available to the workstation when the i.MX 8M Plus serial cable is connected to it. NOTE: you may require to install addtitional COM drivers if you are running on Windows. I like doing the previous step so I can see the result of the next commands issued in U-boot to load the M7 image. fatload mmc 1:1 0x48000000 imx8mp_m7_TCM_sai_low_power_audio.bin; cp.b 0x48000000 0x7e0000 20000; bootaux 0x7e0000 Here is an screenshot that shows how the U-Boot's response should look: U-Boot response when loading the SAI low power audio example to the Cortex M7 That should have prompted the following message on the M7 terminal: M7-core is up!   Now, let’s move to user space! u-boot=> boot 7. Testing the example using a simple GStreamer pipeline   As soon as the O.S. finishes booting. We can see that M7 terminal prompts the following: M7 is now in STOP mode; waiting for some audio to beat the room! Confirm that the WM8960 is listed as audio card as follows: cat /proc/asound/cards             Listing avaialable audio cards. WM8960 should be present. Make note of the list. The wm8960 is listed a the third sound card. This is where I like to differ a bit from [2] and I suggest a quicker test in case of not having an audio file ready. We just simply use GStreamer to play an audiotest source. Please make sure to plug in your headphones onto the board’s 3.5 mm jack before.   The following GStreamer pipeline is using the WM8960 as an audiosink.  gst-launch-1.0 audiotestsrc ! alsasink device=hw:3   NOTE: please be cautious and not put the headphones directly in your head at the first attempt. The sound can be too loud to some people. This is what you should see on the M7 side: Stop the GStreamer pipeline issuing CTRL + C. M7 shall warn you about that: NOTE: you can use the aplay command to play audio as shown on [2]. However, I consider using a testsrc is much quicker and flexible for a quick test.  8. Additional information   Feel free to go ahead and tweak the GStreamer pipeline to change audio test source properties. audiotest src. This command will let you know the available options:            gst-inspect-1.0 audiotestsrc                         NOTE: you can navigate through the displayed list using the “d”key. Press “q’’ to quit. For example:     For example, I am reproducing sound using a different setup based on the list above: gst-launch-1.0 audiotestsrc freq=4000 volume=0.8 wave=8 ! alsasink device=hw:3 9.  Errata and future updates   TBD:     Add an example on how to define the default audio card and play the audio either using gst-play or building the pipeline using filesrc Comment on the limitations of the M7 core regarding sample rate and audio formats  

This is based on L5.10.35 BSP where you have to install QT static build: Qt 5.15 static build: Assuming your sysroot is at "/sysroot-cross" and your toolchain is at "/Toolchain" your qt-source is at /Qt-5.15 PATH=/sysroot-cross/bin:/sysroot-cross/sbin:/Toolchain/bin mkdir /Qt-5.15/mkspecs/qws/linux-imx6-g++ create in this dir the textfile "qmake.conf" with this content: ####################### snip qmake.conf ############################## include(../../common/linux.conf) include(../../common/qws.conf) # modifications to g++.conf QMAKE_CC                = arm-linux-gnueabi-gcc QMAKE_CFLAGS            = -pipe -isystem /sysroot-cross/include -isystem /sysroot-cross/usr/include QMAKE_CXX               = arm-linux-gnueabi-g++ QMAKE_CXXFLAGS          = -pipe -isystem /sysroot-cross/include -isystem /sysroot-cross/usr/include QMAKE_INCDIR            = /sysroot-cross/include /sysroot-cross/usr/include QMAKE_LIBDIR            = /sysroot-cross/lib /sysroot-cross/usr/lib QMAKE_LINK              = arm-linux-gnueabi-g++ QMAKE_LINK_SHLIB        = arm-linux-gnueabi-g++ QMAKE_LFLAGS            = -L/sysroot-cross/lib -L/sysroot-cross/usr/lib -Wl,-rpath-link -Wl,/sysroot-cross/lib QMAKE_LFLAGS           += -Wl,-rpath-link -Wl,/sysroot-cross/usr/lib #Opengl QMAKE_INCDIR_OPENGL = /Vivante/include QMAKE_INCDIR_OPENGL += /Vivante/include/GL QMAKE_INCDIR_OPENGL += /Vivante/include/EGL QMAKE_INCDIR_OPENGL += /Vivante/include/GLES2 QMAKE_LIBDIR_OPENGL = /Vivante/lib QMAKE_INCDIR_OPENGL_ES1 = $$QMAKE_INCDIR_OPENGL QMAKE_LIBDIR_OPENGL_ES1 = $$QMAKE_LIBDIR_OPENGL QMAKE_INCDIR_OPENGL_ES1CL = $$QMAKE_INCDIR_OPENGL QMAKE_LIBDIR_OPENGL_ES1CL = $$QMAKE_LIBDIR_OPENGL QMAKE_INCDIR_OPENGL_ES2 = /Vivante/include QMAKE_INCDIR_OPENGL_ES2 += /Vivante/include/EGL QMAKE_INCDIR_OPENGL_ES2 += /Vivante/include/GLES2 QMAKE_LIBDIR_OPENGL_ES2 = $$QMAKE_LIBDIR_OPENGL QMAKE_INCDIR_EGL = $$QMAKE_INCDIR_OPENGL_ES2 QMAKE_LIBDIR_EGL = $$QMAKE_LIBDIR_OPENGL QMAKE_LIBS_EGL = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL_ES2 = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL_QT = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL_ES1 = QMAKE_LIBS_OPENGL_ES1CL = # modifications to linux.conf QMAKE_AR                = arm-linux-gnueabi-ar cqs QMAKE_OBJCOPY           = arm-linux-gnueabi-objcopy QMAKE_STRIP             = arm-linux-gnueabi-strip QMAKE_CFLAGS_RELEASE   = -pipe -isystem /sysroot-cross/include -isystem /sysroot-cross/usr/include load(qt_config) ####################### snip qmake.conf ############################## create in the same dir the text file "qplatformdefs.h" ####################### snip qplatformdefs.h ############################## #include "../../linux-g++/qplatformdefs.h" ####################### snip qplatformdefs.h ############################## now goto dir /Qt-5.15 cd /Qt-5.15 call configure with ./configure -opensource -confirm-license -release -no-rpath -no-fast \     -no-sql-ibase -no-sql-mysql -no-sql-odbc -no-sql-psql -no-sql-sqlite2 \     -no-qt3support -no-mmx -no-3dnow -no-sse -no-sse2 -no-sse3 -no-ssse3 \     -no-sse4.1 -no-sse4.2 -no-avx -no-optimized-qmake -no-nis -no-cups -pch \     -reduce-relocations -force-pkg-config -prefix /usr -no-armfpa -make libs \     -nomake docs -little-endian -embedded armv6 -qt-decoration-styled \     -depths all -xplatform qws/linux-imx6-g++ -iconv -largefile -qt-gfx-linuxfb \     -qt-gfx-multiscreen -qt-mouse-pc -qt-mouse-linuxinput -qt-libpng \     -plugin-gfx-directfb -system-zlib -no-accessibility -no-gfx-transformed \     -no-gfx-qvfb -no-gfx-vnc -no-kbd-tty -no-kbd-linuxinput -no-kbd-qvfb \     -no-mouse-linuxtp -no-mouse-tslib -no-mouse-qvfb -no-libmng -no-libtiff \     -no-gif -no-libjpeg -no-freetype -no-stl -no-glib -no-openssl -no-egl \     -no-xmlpatterns -no-exceptions -no-multimedia -no-audio-backend -no-phonon \     -no-phonon-backend -no-webkit -no-script -no-scripttools -no-svg -no-script \     -no-declarative -no-sql-sqlite -no-qdbus -no-opengl -static -nomake tools \     -nomake examples -nomake demos when configuring is finished call make after a looong time, when everything goes right, we have a staticly compiled Qt. DO NOT call "make install". We will install manually: copy from /Qt-5.15/bin the files moc, uic, rcc and qmake to somewhere in PATH, eg. /sysroot-cross/bin copy the contents of dir /Qt-5.15/mkspecs to /sysroot-cross/usr/mkspec copy the contents of dir /Qt-5.15/plugins to /sysroot-cross/usr/plugins copy the contents of dir /Qt-5.15/include to /sysroot-cross/usr/include copy the contents of dir /Qt-5.15/lib to /sysroot-cross/usr/lib Test application camtest: if you don't have/want directfb plugin remove from camtest.pro the lines LIBS += -L/sysroot-cross/usr/plugins/gfxdrivers QTPLUGIN += QDirectFBScreen and the lines from main.cpp #include <QtPlugin> Q_IMPORT_PLUGIN(qdirectfbscreen) generate makefile by typing /sysroot-cross/bin/qmake -spec /sysroot-cross/usr/mkspecs/qws/linux-imx6-g++ camtest.pro then make you should set and activate your framebuffers with this script ################# snip ################################ fbset -fb /dev/fb0 -g 1024 768 1024 2304 16 echo -n 0 > /sys/class/graphics/fb0/blank fbset -fb /dev/fb1 -g 1024 768 1024 1536 32 echo -n 0 > /sys/class/graphics/fb1/blank modprobe galcore modprobe uvcvideo modprobe mxc_v4l2_capture ################# snip ################################ if you use directfb then your /etc/directfbrc file should look like this: ######################## snip /etc/directfbrc ############# system=fbdev fbdev=/dev/fb1 mode=1024x768 depth=32 pixelformat=ARGB no-cursor window-surface-policy=systemonly ######################## snip /etc/directfbrc ############# to start the application with directfb: ./camtest -qws -display directfb without directfb using linuxfb: ./camtest -qws -display linuxfb:/dev/fb1 Notes about application: 1. The application shows 2 webcams in background-framebuffer (BG-FB). The foreground-framebuffer (FG-FB) shows the qt-gui. FG-FB is configured to be fully opaque and uses color-keying. On the BG-FB one cam is overlayed on the other cam using IPU. Optimization possibilities: the app copies the frames from the cams with memcpy. This wouldn't be necessary, when the kernel usb-webcam interface (uvc) would support V4L2_MEMORY_USERPTR method. through this way, you could pass the mapped IPU mmapped inbufs directly to v4l2 output buffers. If you get errors like NOSPC (-28) from uvc, this is a limitation of USB. My board is a MX6QSabre, where the two webcams are connected to the same usb-controller. With both webcams I had to limit the frame size to 320x250 and 160x120 at 25Hz. You might try higher res if you have other type of webcams (not usb). Have fun  

This note show how to use the open source gstreamer1.0-rtsp-server package on i.MX6QDS and i.MX8x to stream video files and camera using RTP protocol.  The i.MX 6ULL and i.MX 7 doesn't have Video Processing Unit (VPU). Real Time protocol is a very common network protocol for delivering media over IP networks. On the board, you will need a GStreamer pipeline that encodes the raw video, adds the RTP payload, and sends over a network sink. A generic pipeline would look as follows: video source ! video encoder ! RTP payload ! network sink Video source: often it is a camera, but it can be a video from a file or a test pattern, for example. Video encoder: a video encoder as H.264, H.265, VP8, JPEG and others. RTP payload: an RTP payload that matches the video encoder. Network sink: a video sync that streams over the network, often via UDP.   Prerequisites: MX6x o MX8x board with the L5.10.35 BSP installed. A host PC with either Gstreamer or VLC player installed. Receiving h.264/h.265 Encoded RTP Video Stream on a Host Machine Using GStreamer GStreamer is a low-latency method for receiving RTP video. On your host machine, install Gstreamer and send the following command: $ gst-launch-1.0 -v udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false   Using Host PC: VLC Player Optionally, you can use VLC player to receive RTP video on a PC. First, in your PC, create a sdp file with the following content:  stream.sdpv=0m=video 5000 RTP/AVP 96c=IN IP4 127.0.0.1a=rtpmap:96 H264/90000 After this, with the GStreamer pipepline on the device running, open this .sdp file with VLC Player on the host PC. Sending h.264 and h.265 Encoded RTP Video Stream GStreamer provides an h.264 encoding element by software named x264enc. Use this plugin if your board does not support h.264 encoding by hardware or if you want to use the same pipeline on different modules. Note that the video performance will be lower compared with the plugins with encoding accelerated by hardware. # gst-launch-1.0 videotestsrc ! videoconvert ! x264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000 Note: Replace <host-machine-ip> by the IP of the host machine. In all examples you can replace videotestsrc by v4l2src element to collect a stream from a camera   i.MX8X # gst-launch-1.0 videotestsrc ! videoconvert ! v4l2h264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000   i.MX 8M Mini Quad/ 8M Plus # gst-launch-1.0 videotestsrc ! videoconvert ! vpuenc_h264 ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000 i.MX6X The i.MX6QDS does not support h.265 so the h.264 can work: # gst-launch-1.0 videotestsrc ! videoconvert ! vpuenc_h264 ! rtph264pay config-interval=1 pt=96 ! udpsink host=<host-machine-ip> port=5000   Using Other Video Encoders While examples of streaming video with other encoders are not provided, you may try it yourself. Use the gst-inspect tool to find available encoders and RTP payloaders on the board: # gst-inspect-1.0 | grep -e "encoder"# gst-inspect-1.0 | grep -e "rtp" -e " payloader" Then browse the results and replace the elements in the original pipelines. On the receiving end, you will have to use a corresponding payload. Inspect the payloader element to find the corresponding values. For example: # gst-inspect-1.0 rtph264pay   Install rtp in your yocto different form L5.10.35 BSP, to install gstreamer1.0-rtsp-server in any Yocto Project image, please follow the steps below: Enable meta-multimedia layer: Add the following on your build/conf/bblayers.conf: BBLAYERS += "$"${BSPDIR}/sources/meta-openembedded/meta-multimedia" Include gstreamer1.0-rtsp-server into the image: Add the following on your build/conf/local.conf: IMAGE_INSTALL_append += "gstreamer1.0-rtsp-server" Run bitbake and mount your sdcard. Copy the binaries: Access the gstreamer1.0-rtsp-server examples folder: $ cd /build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer1.0-rtsp-server/$version/build/examples/.libs Copy the test-uri and test-launch to the rootfs /usr/bin folder. $ sudo cp test-uri test-launch /media/USER/ROOTFS_PATH/usr/bin Be sure that the IPs are correctly set: SERVER: => ifconfig eth0 $SERVERIP CLIENT: => ifconfig eth0 $CLIENTIP Video file example SERVER: => test-uri file:///home/root/video_file.mp4 CLIENT: => gst-launch-1.0 playbin uri=rtsp://$SERVERIP:8554/test You can try to improve the framerate performance using manual pipelines in the CLIENT with the rtspsrc plugin instead of playbin. Follow an example: => gst-launch-1.0 rtspsrc location=rtsp://$SERVERIP:8554/test caps = 'application/x-rtp'  ! queue max-size-buffers=0 ! rtpjitterbuffer latency=100 ! queue max-size-buffers=0 ! rtph264depay ! queue max-size-buffers=0 ! decodebin ! queue max-size-buffers=0 ! imxv4l2sink sync=false   Camera example SERVER: => test-launch "( imxv4l2src device=/dev/video0 ! capsfilter caps='video/x-raw, width=1280, height=720, framerate=30/1, mapping=/test' ! vpuenc_h264 ! rtph264pay name=pay0 pt=96 )" CLIENT: => gst-launch-1.0 rtspsrc location=rtsp://$SERVERIP:8554/test ! decodebin ! autovideosink sync=false The rtspsrc has two properties very useful for RTSP streaming: Latency: Useful for low-latency RTSP stream playback (default 200 ms); Buffer-mode: Used to control buffer mode. The slave mode is recommended for low-latency communications. Using these properties, the example below gets 29 FPS without a sync=false property in the sink plugin. The key achievement here is the fact that there is no dropped frame: => gst-launch-1.0 rtspsrc location=rtsp://$SERVERIP:8554/test latency=100 buffer-mode=slave ! queue max-size-buffers=0 ! rtph264depay ! vpudec ! imxv4l2sink      

This document is a user guide for the GStreamer version 1.0 based accelerated solution included in all the i.MX 8 family SoCs supported by NXP BSP L5.4.24_1.1.0. Some instructions assume a host machine running a Linux distribution, such as Ubuntu, connected to i.MX 8 device. These commands were tested using Ubuntu 18.04 LTD, and while Ubuntu is not required on the host machine, other distributions have not been tested. These instructions are targeted for use with the following hardware: • i.MX 8MQ EVK • i.MX 8MN EVK • i.MX 8MN EVK • i.MX 8QXP MEK B0 • i.MX 8QM MEK B0   Release History v1.0 - Mar 2020 - Initial release. v2.0 - Sep 2020: Added the following content: - Mux/Demux Examples - Audio Examples - Image Examples - Transcode Examples - Streaming Examples - Multi-Display Examples - Scaling and Rotation Examples - Zero-copy Examples - Debug Examples Maintainers: . Marco Franchi . Pedro Jardim

In order to get USB cameras (web cams) working on i.MX 51 EVK board running Ubuntu, a few steps must be followed, and they are: Enable USB Camera's drivers on Kernel Test it using Gstreamer or another compatible software (as Cheese) Kernel Driver USB cameras (web cameras) on Linux work over GSPCA driver, to enable this driver you need to go to: ./ltib -c   [*] Configure the kernel     Device Drivers -->          Multimedia Devices -->               [*] Video Capture Adapters -->                    [*] V4L USB Devices -->                         <*> USB Video Class (UVC)                                      [*] UVC input events device support                         <*> GSPCA Based WebCams --> From this point, you need to choose your specific driver. If you don't know, you can select all of those options as a built-in module "<*>" that will work. GSPCA Drivers USB Camera Detection Connect your USB camera to the USB Host port on i.MX 51 EVK board and then type "dmesg", and also check if there is a video0 device using: ubuntu@ubuntu-desktop:~$ ls /dev/video0 /dev/video0 USB Camera Detection Gstreamer Command Line In order to test your USB camera using Gstreamer plugin, use the following command line to perform it: ubuntu@ubuntu-desktop:~$ gst-launch-0.10 v4l2src ! ffmpegcolorspace ! ximagesink and the results: Hi there !!! EOF !

The i.MX 8QXP MEK does not allow the OV5640/LVDS/LCD usage only by changing the device tree anymore. It occurs because the M4 owns the i2c resources, so the A core must use rpmsg to enable virtual drivers. Due to this, if the user changes the device tree, for instance, the *ov5640.dtb, the kernel won't boot, entering in the following loop: [    8.603353] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).      [    8.610025] [drm] No driver support for vblank timestamp query.              [    8.616077] imx-drm display-subsystem: bound imx-drm-dpu-bliteng.2 (ops dpu_) [    8.624978] imx-drm display-subsystem: bound imx-dpu-crtc.0 (ops dpu_crtc_op) [    8.632526] imx-drm display-subsystem: bound imx-dpu-crtc.1 (ops dpu_crtc_op) [    8.639833] imx-drm display-subsystem: failed to bind ldb@562210e0 (ops imx_7 [    8.648428] imx-drm display-subsystem: master bind failed: -517 With the approach provided in this post, it is possible to make this change manually, only by changing the flash.bin at U-boot for a non-m4 one. In order to make the changes to the flash.bin file, it’s needed to obtain the following files: - u-boot.bin from internal u-boot provided by NXP. - scfw_tcm.bin from SCFW porting kit - bl31.bin from ARM Trusted Firmware - SECO firmware container image Disclaimer The described procedures in this document target a GNU/Linux (Ubuntu 20.04 LTS) and it’s focused on iMX8QXP B0 + BSP L4.19.35_1.1.0. Required packages 1 - Install ARM64 ToolChain: 1.1 - Install ARM64 GCC and G++ cross-compilers: # apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu 2 - Install ARM32 GCC6 ToolChain: 2.1 - Download the ARM32 6 Toolchain and install it: $ mkdir ~/gcc_toolchain $ cp ~/Downloads/gcc-arm-none-eabi-6-2017-q2-update-linux.tar.bz2 ~/gcc_toolchain/ $ cd ~/gcc_toolchain/ $ tar xvjf gcc-arm-none-eabi-6-2017-q2-update-linux.tar.bz2 # apt-get update # apt-get install srecord 3 - Download MKimage 3.1 - Create a new directory desired to the packages: $ mkdir flash_build $ cp flash_build 3.1 - Clone the MKimage: $ git clone https://source.codeaurora.org/external/imx/imx-mkimage -b imx_4.19.35_1.1.0 4 - U-boot build 4.1 - Clone the U-boot  $ git clone https://source.codeaurora.org/external/imx/uboot-imx -b imx_v2019.04_4.19.35_1.1.0 $ cd uboot-imx 4.2 - Export the ARM64 ToolChain:  $ export ARCH=arm64 $ export CROSS_COMPILE=/usr/bin/aarch64-linux-gnu- 4.3 - Build it:  $ unset LDFLAGS $ make -j4 imx8qxp_mek_defconfig $ make 4.4 - Copy the binary files to the MKimage/iMX8QX directory:  $ cp spl/u-boot-spl.bin ../imx-mkimage/iMX8QX/ $ cp u-boot-nodtb.bin ../imx-mkimage/iMX8QX/ $ cd ..   5 - ARM Trusted Firmware 5.1 - Clone the imx-atf:  $ git clone https://source.codeaurora.org/external/imx/imx-atf -b imx_4.19.35_1.1.0 $ cd imx-atf 5.2 - Build it:  $ unset LDFLAGS $ make PLAT=imx8qx bl31 5.3 - Copy the binary files to the MKimage/iMX8QX directory:  $ cp build/imx8qx/release/bl31.bin ../imx-mkimage/iMX8QX/ $ cd ..   6 - SCFW 6.1 - Export the ARM32 GCC6 Toolchain:  $ export TOOLS=~/gcc_toolchain/ 6.2 - Download the BSP L4.19.35_1.1.0_SCFW and copy it to the flash_build directory:  $ cp ~/Downloads/imx-scfw-porting-kit-1.2.7.1.tar.gz $ tar xvzf imx-scfw-porting-kit-1.2.7.1.tar.gz $ cd packages/ $ chmod a+x imx-scfw-porting-kit-1.2.7.1.tar.gz $ ./imx-scfw-porting-kit-1.2.7.1.bin 6.3 - Build it to i.MX 8QXP MEK B0:  $ cd imx-scfw-porting-kit-1.2.7.1/src/ $ tar xvzf scfw_export_mx8qx_b0.tar.gz $ cd scfw_export_mx8qx_b0/ $ make qx R=B0 B=mek 6.4 - Copy the binary file to the MKimage/iMX8QX directory:  $ cp build_mx8qx_b0/scfw_tcm.bin ../../../../imx-mkimage/iMX8QX/ $ cp ../../../../ 7 - SECO Firmware Container Image 7.1 - Download the SECO firmware binaries and copy it to the flash_build directory $ cp ~/Downloads/firmware-imx-7.9.bin . $ chmod a+x firmware-imx-7.9.bin 7.2 - Copy the binary files to the MKimage/iMX8QX directory:  $ cp firmware-imx-7.9/firmware/seco/mx8qx-ahab-container.img /imx-mkimage/iMX8QX/ 8 - Build flash.bin 8.1 - In a new terminal, open the imx-mkimage directory: $ cd flash_build/imx-mkimage 8.2 - Build it:  $ make SOC=iMX8QX flash 8.3 - Deploy it to the SDCard:  $ sudo dd if=iMX8QX/flash.bin of=/dev/sdX bs=1k seek=32 && sync Now, you are able to use any non-rpmsg.dtb without kernel errors. Author: Pedro Jardim: pedro.jardim@nxp.com

Recently I published this i.MX Dev Blog post about the Gateworks plugin gst-variable-rtsp-server support for i.MX 6. Now, you can check how to use it on i.MX 8 SoCs as well. 1. Preparing the image In order to use gst-variable-rtsp-server plugin, prepare your machine and distro: Add the following line to conf/local.conf: IMAGE_INSTALL_append += "gstreamer1.0-rtsp-server gst-variable-rtsp-server" Download the attached patch and apply it by doing: $ cd <yocto_path>/sources/meta-fsl-bsp-release/ $ git am ~/Download/0001-Add-RTSP-support-for-i.MX-8-L4.14.78_ga1.0.0-or-olde.patch Note: This patch is not necessary for L4.14.98_ga2.0.0 BSP! Then, build the image with bitbake and deploy it to the SD card. 2. Video Test Source Example Server $ gst-variable-rtsp-server -p 9001 -u "videotestsrc ! v4l2h264enc ! rtph264pay name=pay0 pt=96" Client 2. Camera Example Server $ gst-variable-rtsp-server -p 9001 -u "v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! v4l2h264enc ! rtph264pay name=pay0 pt=96" Client In order to use VLC or other application as the client, just enter the URL as shown in the image below:

///////////////////////////create device node /dev/galcore///////////////////////////// $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/Kbuild MODULE_NAME ?= galcore /* define node name*/ $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/hal/os/linux/kernel/gc_hal_kernel_linux.h define DEVICE_NAME "galcore" $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/hal/os/linux/kernel/gc_hal_kernel_probe.c drv_init call ret = register_chrdev(major, DEVICE_NAME, &driver_fops); ///////////////////////////////opengles2 functios/////////////////////////////////////////// myandroid/device/fsl-proprietary/gpu-viv/lib/egl/libGLESv2_VIVANTE.so glActiveTexture glBindBuffer ... ... ... //those glxxxxxx call into sub_D40C int __fastcall sub_D40C(int a1, int a2, int a3) //address 0x0000D40C { int result; // r0@1 int v4; int v5; v4 = a2;   v5 = a3;   gcoOS_GetTLS(&v4);  //------------> goto libGAL.so   result = v4;   if ( v4 )     result = *(_DWORD *)(v4 + 36);   return result; } and $home/myandroid/device/fsl-proprietary/gpu-viv/lib/libGAL.so //export function signed int __fastcall gcoOS_GetTLS(void **a1) { ... ... gcoOS_GetTLS v4 = open("/dev/galcore", 2); ... ... } and device node /dev/galcore pass command into module galcore $home/myandroid/kernel_imx/drivers/mxc/gpu-viv/hal/kernel/gc_hal_kernel.c gckKERNEL_Dispatch This document was generated from the following discussion: Share Vivante 3d gc2000 work flow

Check new updated version for with Morty here Step 1 : Get iMX Yocto AVS setup environment Review the steps under Chapter 3 of the i.MX_Yocto_Project_User'sGuide.pdf on the L4.X LINUX_DOCS to prepare your host machine. Including at least the following essential Yocto packages $ sudo apt-get install gawk wget git-core diffstat unzip texinfo \   gcc-multilib build-essential chrpath socat libsdl1.2-dev u-boot-tools Install the i.MX NXP AVS repo Create/Move to a directory where you want to install the AVS yocto build enviroment. Let's call this as <yocto_dir> $ cd <yocto_dir> $ repo init -u https://source.codeaurora.org/external/imxsupport/meta-avs-demos -b master -m imx7d-pico-avs-sdk_4.1.15-1.0.0.xml Download the AVS BSP build environment: $ repo sync Step 2: Setup yocto for Alexa_SDK image with AVS-SETUP-DEMO script: Run the avs-setup-demo script as follows to setup your environment for the imx7d-pico board: $ MACHINE=imx7d-pico DISTRO=fsl-imx-x11 source avs-setup-demo.sh -b <build_sdk> Where <build_sdk> is the name you will give to your build folder. After acepting the EULA the script will prompt if you want to enable: Sound Card selection The following Sound Cards are supported on the build: SGTL (In-board Audio Codec for PicoPi) 2-Mic Conexant The script will prompt if you are going to use the Conexant Card. If not then SGTL will be assumed as your selection Are you going to use Conexant Sound Card [Y/N]? Install Alexa SDK Next option is to select if you want to pre-install the AVS SDK software on the image. Do you want to build/include the AVS_SDK package on this image(Y/N)? If you select YES, then your image will contain the AVS SDK ready to use (after authentication). Note this AVS_SDK will not have WakeWord detection support, but it can be added on runtime. If your selection was NO, then you can always manually fetch and build the AVS_SDK on runtime. All the packages dependencies will be already there, so only fetching the AVS_SDK source code and building it is required. Finish avs-image configuration At the end you will see a text according with the configuration you select for your image build. Next is an example for a Preinstalled AVS_SDK with Conxant Sound Card support and WiFi/BT not enabled. ==========================================================   AVS configuration is now ready at conf/local.conf             - Sound Card = Conexant                                     - AVS_SDK pre-installed                                       You are ready to bitbake your AVS demo image now:               bitbake avs-image                                        ========================================================== Step 3: Build the AVS image Go to your <build_sdk> directory and start the build of the avs-image There are 2 options Regular Build: $ cd <yocto_dir>/<build_sdk> $ bitbake avs-image With QT5 support included: $ cd <yocto_dir>/<build_sdk> $ bitbake avs-image-qt5 The image with QT5 is useful if you want to add some GUI for example to render DisplayCards. Step 4 : Deploying the built images to SD/MMC card to boot on target board. After a build has succesfully completed, the created image resides at <build_sdk>/tmp/deploy/images/imx7d-pico/ In this directory, you will find the imx7d-pico-avs.sdcard image or imx7d-pico-avs-qt5.sdcard, depending on the build you chose on Step3. To Flash the .sdcard image into the eMMC device of your PicoPi board follow the next steps: Download the bootbomb flasher Follow the instruction on Section 4. Board Reflashing of the Quick Start Guide for AVS kit to setup your board on flashing mode. Copy the built SDCARD file $ sudo dd if=imx7d-pico-avs.sdcard of=/dev/sd bs=1M && sync $ sync Properly eject the pico-imx7d board: $ sudo eject /dev/sd NXP Documentation Refer to the Quick Start Quide for AVS SDK to fully setup your PicoPi board with Synaptics 2Mic and PicoPi i.mx7D For a more comprehensive understanding of Yocto, its features and setup; more image build and deployment options and customization, please take a look at the i.MX_Yocto_Project_User's_Guide.pdf document from the Linux documents bundle mentioned at the beginning of this document. For a more detailed description of the Linux BSP, u-boot use and configuration, please take a look at the i.MX_Linux_User's_Guide.pdf document from the Linux documents bundle mentioned at the beginning of this document.