One of the most popular use cases for embedded systems are projects destinated to show information and interact with users. These views are called GUI or Graphic User Interface which are designed to be intuitive, attractive, consistent, and clear. There are many tools that we can use to achieve great GUIs, mostly implemented for platforms such as Web, Android, and iOS. Here, we will need to introduce the concept of framework, basically, it is a set of tools and rules that provides a minimal structure to start with your development. Frameworks usually comes with configuration files, code snippets, files and folders organization helping us to save time and effort. Also, it is important to review the concept of SDK or Software Development Kit which is a set of tools that allows to build software for specific platforms. Usually supplies debugging tools, documentation, libraries, API’s, emulators, and sample code. Flutter is an open-source UI software development kit by Google that help us to create applications with great GUIs on different platforms from a single codebase. Depends on the reference, you can find Flutter defined as a framework or SDK and both are correct, however, an SDK could be a best definition thanks to Flutter supplies a wide and complete package to create an application in which framework is also included. This article is aimed at those that are in a prototyping stage looking for a different tool to develop projects. Also, this article pretends to be a theoretical introduction explaining the most important concepts. However, is a good practice to learn more about reviewing the official documentation from Flutter. (Flutter documentation | Flutter) Here is the structure used throughout this article: What is Flutter? Flutter details Platforms Programming language Official documentation Flutter for embedded systems What is Flutter? Flutter was officially released by Google in December 2018 with a main aim, to give developers a tool to create applications natively compiled for mobile (Android, iOS), web and desktop (Windows, Linux) from a single codebase. It means that as a developer, Flutter will create a structure with minimal code, configuration files, build files for each operating system, manifests, etc. in which we will add our custom code and finally build this code for our preferred OS. For example, we can create an application to review fruit and vegetable information and compile for Android and iOS with the same code. A basic Flutter development process based on my experience looks like the following diagram: Flutter has the following key features: Cross-platform development. Flutter allows the developer to create applications for different platforms using a single codebase. It means that you will not need to recreate the application for each platform you want to support.   Hot-reload. This feature allows the developer to see changes in real time without restarting the whole application, this results in time savings for your project.   High Performance Flutter apps achieve high performance due to the app code is compiled to native ARM code. With this tool no interpreters are involved.   UI Widgets Flutter supplies a set of widgets (UI components such as boxes, inputs text, buttons, etc.) predefined by UI systems guidelines Material on Android and Cupertino for iOS. Source: Material 3 Design Kit | Figma Community Source: Design - Apple Developer   Great community support. This feature could be subjective but, it is useful when we are developing our project find solutions to known issues or report new ones. Because of Flutter is an open source and is widely implemented in the industry this tool owns a big community, with events, forums, and documentation. Flutter Details Supported Platforms With Flutter you can create applications for: Android iOS Linux Debian Linux Ubuntu macOS web Chrome, Firefox, Safari, Edge Windows Supported deployment platforms | Flutter Programming Language Flutter use Dart, a programming language is an open-source language supported by Google optimized to use on the creation of user interfaces. Dart key features: Statically typed. This feature helps catching errors making the code robust ensuring that the variable’s value always match with the declared variable’s type. Null safety. All variables on Dart are non-nullable which means that every variable must have a non-null value avoiding errors at execution time. This feature also, make the code robust and secure. Async/Await. Dart is client-optimized which means that this language was specially created to ensure the best performance as a client application. Async/Await is a feature part of this optimization making easier to manage network requests and other asynchronous operations. Object oriented. Dart is an object-oriented language with classes and mixin. This is especially useful to use on Flutter with the usage of widgets. Compiler support of Just-In-Time (JIT) and Ahead-of-Time (AOT) JIT provides the support that enables the Hot Reload Flutter feature that I mentioned before. It is a complex mechanism, but Dart “detects” changes in your code and execute only these changes avoiding recompiling all the code. AOT compiler produces efficient ARM code improving start up time and performance. Official documentation Flutter has a rich community and documentation that goes from UI guidelines to an Architectural Overview. You can find the official documentation at the following links: Flutter Official Documentation: Flutter documentation | Flutter Flutter Community: Community (flutter.dev) Dart Official Documentation: Dart documentation | Dart Flutter for embedded systems So far, we know all the excellent features and platforms that Flutter can support. But, what about the embedded systems? On the official documentation we can find that Flutter may be used for embedded systems but in fact there is no an official supported platform. This SDK has been supported by their community, specially there is one repository on GitHub supported by Sony that provides documentation and Yocto recipes to support Flutter on embedded Linux. To understand the reason to differentiate between Flutter for Linux Desktop with official support and to create a specific Flutter support for embedded Linux is important to describe the basics of Flutter architecture. Based on the Flutter documentation the system is designed using layers that can be illustrated as follows:   Source: Flutter architectural overview | Flutter We can see as a top level “Framework” which is a high-level layer that includes widgets, tools and libraries that are in contact with developers. Below “Framework,” the layer “Engine” is responsible of drawing the widgets specified in the previous layer and provides the connection between high-level and low-level code. This layer is mostly written in C++ for this reason Flutter can achieve high performance running applications. Specifically for graphics rendering Flutter implements Impeller for iOS and Skia for the rest of platforms. The bottom layer is “Embedder” which is specific for each target and operating system this layer allows Flutter application to run as a native app providing the access to interact with different services managed by the operating systems such as input, rendering surfaces and accessibility. This layer for Linux Desktop uses GTK/GDK and X11 as backend that is highly dependent of unnecessary libraries and expensive for embedded systems which have constrained resources for computation and memory. The work around founded by Sony’s Flutter for Embedded Linux repository is to change this backend using a widely implemented backend for embedded systems Wayland. The following image illustrates the difference between Flutter for Linux Desktop and Flutter for Embedded Linux.   Source: What's the difference between Linux desktop and Embedded Linux · sony/flutter-embedded-linux Wiki · GitHub   Source: What's the difference between Linux desktop and Embedded Linux · sony/flutter-embedded-linux Wiki · GitHub Here is the link to the mentioned repository: GitHub - sony/flutter-elinux: Flutter tools for embedded Linux (eLinux) Finally, I would like to encourage you to read the official Flutter documentation and consider this tool as a great option compared to widely used tools on embedded devices such as Qt or Chromium. Also, please have a look to a great article written by Payam Zahedi delving into the implementation of Flutter for Embedded Linux measuring performance and giving conclusions about the usage of Flutter in embedded systems. (Flutter on Embedded Devices. Learn how to run Flutter on embedded… | by Payam Zahedi | Snapp Embedded | Medium).    

This is a tool for screen capture under DRM (Direct Render Manager). This also a revised version for previous “drmfbcap” (DRM Framebuffer Capture). Unlike the FB based system under which we can capture the frame buffer easily through reading the device node, the DRM is much more complex and secure-protected. No direct way for reading framebuffer data from user space. Under DRM case, we need to open the DRM device, query the resource, get and map the FB object and then read the buffer eventually. With this tool, we can capture the buffer content from a DRM device and output as raw RGB/YUV data. Features: Capture all planes or specific plane, including hidden/covered planes or planes (overlays) managed by applications directly. Both RGB and YUV supported (auto detect). Tile format (VSI Super-Tile) is also supported. Repeat mode which can capture frames continuously. Tool was built as static linked, in this case, it should be working in both Linux and Android.   Important notes: Behavior of DRM subsystem is different between Linux 4.x and 5.x/6.x. For Linux 4.x, you can capture the RGB buffer without any problem. But, there’s no API for YUV (multi-plane) buffer. To capture YUV, please patch kernel with: “kernel_0001-drm-Add-getfb2-ioctl_L4.14.98.patch”. For Linux 5.x, mapping/capturing the internal buffer is not allowed by default due to security reason. To overcome this temporary (for debug only), patch the kernel with: “0001-drm-enable-mapping-of-internal-object-for-debugging_L5.x.patch”. It contains a minor change to remove this guard. Both patches are included in attachment. To get more details about how to use this tool, try “-h” option to print the usage message. Enjoy!

Customer is asking high-capacity external storage(for example >64GB) support on i.MX BSP, ext4 is ok for HC storage, but it can’t be supported by Windows. Pls find NFTS and exFAT support status on Linux BSP below: Updated test result on L5.4.70.2.3.0 and L6.1.22: L5.4.70.2.3.0 1.You can enable ntfs support in kernel config as below,  ntfs can be mounted normally, but you can only modify existing file content in disk, you can’t create/delete/rename file on disk. > File systems > DOS/FAT/NT Filesystems   Log: root@imx8mpevk:~# mount -t ntfs /dev/sda1 /mnt/fat/ [  662.732869] ntfs: volume version 3.1. root@imx8mpevk:~# cp ntfs-3g /mnt/fat/ cp: cannot create regular file '/mnt/fat/ntfs-3g': Permission denied root@imx8mpevk:~# ls /mnt/fat/ 111.png  Image_org  System Volume Information  gpuinfo.sh root@imx8mpevk:~# vi /mnt/fat/gpuinfo.sh root@imx8mpevk:~# umount /mnt/fat/ root@imx8mpevk:~# ntfs file system can be accessed via ntfs-3g in user space as below //build: wget https://tuxera.com/opensource/ntfs-3g_ntfsprogs-2017.3.23.tgz tar zxvf ntfs-3g_ntfsprogs-2017.3.23.tgz cd ntfs-3g_ntfsprogs-2017.3.23/ source ../../sdk/environment-setup-aarch64-poky-linux   ./configure --host=aarch64-linux --build=aarch64-poky-linux --disable-shared --enable-static   make   ls /src/ntfs-3g   //put it into rootfs cp ntfs-3g /bin   //test log: root@imx8mpevk:/# [ 1058.724471] usb 1-1: USB disconnect, device number 4 [ 1062.058613] usb 1-1: new high-speed USB device number 5 using xhci-hcd [ 1062.214029] usb-storage 1-1:1.0: USB Mass Storage device detected [ 1062.220986] scsi host0: usb-storage 1-1:1.0 [ 1063.235871] scsi 0:0:0:0: Direct-Access     VendorCo ProductCode      2.00 PQ: 0 ANSI: 4 [ 1063.246185] sd 0:0:0:0: [sda] 15728640 512-byte logical blocks: (8.05 GB/7.50 GiB) [ 1063.254023] sd 0:0:0:0: [sda] Write Protect is off [ 1063.259164] sd 0:0:0:0: [sda] No Caching mode page found [ 1063.264540] sd 0:0:0:0: [sda] Assuming drive cache: write through [ 1063.296946]  sda: sda1 [ 1063.300860] sd 0:0:0:0: [sda] Attached SCSI removable disk   root@imx8mpevk:/# ntfs-3g /dev/sda1 /mnt/fat/ root@imx8mpevk:/# ls /mnt/fat/ README  System Volume Information  gpu.sh  gpuinfo.sh root@imx8mpevk:/# cp /unit_tests/memtool /mnt/fat/ root@imx8mpevk:/# umount /mnt/fat/ root@imx8mpevk:/# ntfs-3g /dev/sda1 /mnt/fat/ root@imx8mpevk:/# ls /mnt/fat/ README  System Volume Information  gpu.sh  gpuinfo.sh  memtool root@imx8mpevk:/#   3.exFAT is not supported on this BSP..   L6.1.22(you can check it on L5.15 and above, should be the same) You can enable ntfs support in kernel config as below, full features can be supported. > File systems > DOS/FAT/EXFAT/NT Filesystems   Pls use ‘-t ntfs3’ during mounting, otherwise it will be mounted as ‘read-only’ Log: root@imx8ulpevk:~# mount -t ntfs3 /dev/sda1 /mnt/fat/ root@imx8ulpevk:~# ls /mnt/fat/ 111.png   Image_org  'System Volume Information' root@imx8ulpevk:~# root@imx8ulpevk:~# cp gpuinfo.sh /mnt/fat/ root@imx8ulpevk:~# umount /mnt/fat/ root@imx8ulpevk:~# root@imx8ulpevk:~# mount -t ntfs3 /dev/sda1 /mnt/fat/ root@imx8ulpevk:~# ls /mnt/fat/ 111.png   Image_org  'System Volume Information'   gpuinfo.sh root@imx8ulpevk:~#   exFAT has been supported in L6.1.22. > File systems > DOS/FAT/EXFAT/NT Filesystems   /dev/sda1 on /run/media/sda1 type exfat (rw,relatime,fmask=0022,dmask=0022,iocharset=utf8,errors=remount-ro) root@imx8ulpevk:~# ls /run/media/sda1 'Certificate of Completion.pdf'             carlife.MP4 Image_org                                  example.tflite L5.4.70_2.3.0                              mx8mp_vpu.txt NXP-5G.mp4                                 sd.mp4 'System Volume Information'                 vela.ini android_p9.0.0_2.1.0-auto-ga_image_8qmek root@imx8ulpevk:~# ls Image_org  gpuinfo.sh root@imx8ulpevk:~# cp gpuinfo.sh /run/media/sda1/ root@imx8ulpevk:~# umount /run/media/sda1 root@imx8ulpevk:~#

In some cases, such as mass production or preparing a demo. We need u-boot environment stored in demo sdcard mirror image.  Here is a way: HW:  i.MX8MP evk SW:  LF_v5.15.52-2.1.0_images_IMX8MPEVK.zip The idea is to use fw_setenv to set the sdcard mirror as the operation on a real emmc/sdcard. Add test=ABCD in u-boot-initial-env for test purpose. And use fw_printenv to check and use hexdump to double confirm it. The uboot env is already written into sdcard mirror(imx-image-multimedia-imx8mpevk.wic). All those operations are on the host x86/x64 PC. ./fw_setenv -c fw_env.config -f u-boot-initial-env Environment WRONG, copy 0 Cannot read environment, using default ./fw_printenv -c fw_env.config Environment OK, copy 0 jh_root_dtb=imx8mp-evk-root.dtb loadbootscript=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${bsp_script}; mmc_boot=if mmc dev ${devnum}; then devtype=mmc; run scan_dev_for_boot_part; fi arch=arm baudrate=115200 ...... ...... ...... splashimage=0x50000000 test=ABCD usb_boot=usb start; if usb dev ${devnum}; then devtype=usb; run scan_dev_for_boot_part; fi vendor=freescale hexdump -s 0x400000 -n 2000 -C imx-image-multimedia-imx8mpevk.wic 00400000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| hexdump -s 0x400000 -n 10000 -C imx-image-multimedia-imx8mpevk.wic 00400000 5f a4 9b 97 20 6a 68 5f 72 6f 6f 74 5f 64 74 62 |_... jh_root_dtb| 00400010 3d 69 6d 78 38 6d 70 2d 65 76 6b 2d 72 6f 6f 74 |=imx8mp-evk-root| 00400020 2e 64 74 62 00 20 6c 6f 61 64 62 6f 6f 74 73 63 |.dtb. loadbootsc| 00400030 72 69 70 74 3d 66 61 74 6c 6f 61 64 20 6d 6d 63 |ript=fatload mmc| 00400040 20 24 7b 6d 6d 63 64 65 76 7d 3a 24 7b 6d 6d 63 | ${mmcdev}:${mmc| 00400050 70 61 72 74 7d 20 24 7b 6c 6f 61 64 61 64 64 72 |part} ${loadaddr| 00400060 7d 20 24 7b 62 73 70 5f 73 63 72 69 70 74 7d 3b |} ${bsp_script};| 00400070 00 20 6d 6d 63 5f 62 6f 6f 74 3d 69 66 20 6d 6d |. mmc_boot=if mm| ...... ...... ...... 00401390 76 3d 31 00 73 6f 63 3d 69 6d 78 38 6d 00 73 70 |v=1.soc=imx8m.sp| 004013a0 6c 61 73 68 69 6d 61 67 65 3d 30 78 35 30 30 30 |lashimage=0x5000| 004013b0 30 30 30 30 00 74 65 73 74 3d 41 42 43 44 00 75 |0000.test=ABCD.u| 004013c0 73 62 5f 62 6f 6f 74 3d 75 73 62 20 73 74 61 72 |sb_boot=usb star| 004013d0 74 3b 20 69 66 20 75 73 62 20 64 65 76 20 24 7b |t; if usb dev ${| 004013e0 64 65 76 6e 75 6d 7d 3b 20 74 68 65 6e 20 64 65 |devnum}; then de| flash the sdcard mirror into i.MX8MP evk board emmc to check uuu -b emmc_all imx-boot-imx8mp-lpddr4-evk-sd.bin-flash_evk imx-image-multimedia-imx8mpevk.wic  The first time boot, the enviroment is already there.  How to achieve that: a. fw_setenv/fw_printenv: https://github.com/sbabic/libubootenv.git Note: Please do not use uboot fw_setenv/fw_printenv Compile it on the host x86/x64 PC. It is used on host. b. u-boot-initial-env Under uboot, make u-boot-initial-env Note: Yocto deploys u-boot-initial-env by default c. fw_env.config  imx-image-multimedia-imx8mpevk.wic 0x400000 0x4000 0x400000 0x4000 are from uboot-imx\configs\imx8mp_evk_defconfig CONFIG_ENV_SIZE=0x4000 CONFIG_ENV_OFFSET=0x400000 Now, you can run  ./fw_setenv -c fw_env.config -f u-boot-initial-env

  Platform: i.MX8MP EVK , L6.1.22-2.0.0 LT9211 is a chip that can realize the conversion of MIPI DSI signals to LVDS signals. This patch is based on this mainline driver:https://github.com/nxp-imx/linux-imx/blob/lf-6.1.y/drivers/gpu/drm/bridge/lontium-lt9211.c Keypoint Move lt9211_host_attach function to lt9211_attach to skip bridge attach error.  

Traditional non-matter devices cannot directly join the matter network. But Matter Bridge solves the problem. Matter bridge can join a Matter network as a Matter device and nonmatter devices need to be mapped to Matter network as a dynamic endpoint. In this way, other Matter devices can communicate with non-matter devices through dynamic endpoints. The Guide is a Matter Zigbee Bridge implement based on i.MX93 + K32W0.     Feature List • Matter over Ethernet • Matter over Wi-Fi • Register and Remove Zigbee Deivces • Connect Zigbee devices into Matter ecosystem seamlessly • Zigbee Devices o OnOff cluster o Temperature Sensor Cluster • Matter Actions o Start Zigbee Network o Zigbee Network Permit Join o Factory Reset • No limitation if migrating to other i.MX MPU like i.MX6ULL, i.MX8MP • OTBR and Zigbee bridge can be integrated into one single device

This demo for all(bootloader, device tree, Linux kernel, rootfs) in spi. It uses raw read(sf read)/raw write(sf write in uuu script) to achieve that. sf probe 0; sf read ${fdt_addr} 0x500000 0x100000; sf read ${loadaddr} 0x600000 0x1E00000; sf read ${initrd_addr} 0x2400000 0x600000; setenv bootargs console=${console},${baudrate} earlycon=${earlycon},${baudrate} rdinit=/linuxrc; booti ${loadaddr} ${initrd_addr} ${fdt_addr} |-- 0001-all-in-spi-demo-lf-5.10.72-2.2.0.patch --- patch for this demo |-- demo_binary | |-- flash.b0.bin --- b0 bootloader | |-- flash.bin --- c0 bootloader | |-- Image-imx8qxpc0mek.bin --- Linux kernel | |-- imx8qxp-mek.dtb --- device tree | |-- uramdisk_boot.rootfs.aarch64.img --- ram disk | |-- uuu.qspi.all.b0.uuu --- uuu script for b0 | `-- uuu.qspi.all.uuu --- uuu script for c0 `-- readme.txt --- this file # The spi layout used is: # - --------- -------------------------------------------- # | | flash.bin | env | dtb | Image |rootfs| # - --------------- -------------------------------------- # ^ ^ ^ ^ ^ ^ ^ # | | | | | | | # 0 4kiB 4MiB 5MiB 6MiB 36MiB 42MiB 0x1000 0x400000 0x500000 0x600000 0x2400000 Test: HW: i.MX8QXP MEK SW: lf-5.10.72-2.2.0 + 0001-all-in-spi-demo-lf-5.10.72-2.2.0.patch Test log: SF: Detected mt35xu512aba with page size 256 Bytes, erase size 128 KiB, total 64 MiB device 0 offset 0x500000, size 0x100000 SF: 1048576 bytes @ 0x500000 Read: OK device 0 offset 0x600000, size 0x1e00000 SF: 31457280 bytes @ 0x600000 Read: OK device 0 offset 0x2400000, size 0x600000 SF: 6291456 bytes @ 0x2400000 Read: OK [ 4.787552] imx6q-pcie 5f010000.pcie: unable to add pcie port. [ 4.797467] Freeing unused kernel memory: 2944K [ 4.807379] Run /linuxrc as init process Starting syslogd: OK Starting klogd: OK Running sysctl: OK Starting network: OK /bin/sh: can't access tty; job control turned off / #  

In this article, I will explain how to set up the iMX8M Plus to use the 4K Dart BCON Basler Camera module. Requirements: Evaluation Kit for the i.MX 8M Plus Applications Processor. (i.MX 8M Plus Evaluation Kit | NXP Semiconductors) Basler Camera for i.MX 8M Plus (4K dart BCON for MIPI camera module for i.MX 8M Plus | NXP Semiconductors). Embedded Linux for i.MX Applications Processors (Embedded Linux for i.MX Applications Processors | NXP Semiconductors) (For this example we will use BSP version Linux 5.15.71_2.2.0) Serial Console Emulator Basler Camera Specifications and Manuals: Basler Camera Specifications at this link: Embedded Vision Kits daA3840-30mc-IMX8MP-EVK - Embedded Vision Kits (baslerweb.com). Basler Manual to identify and setting up the hardware at this link: daA3840-30mc-IMX8MP-EVK | Basler Product Documentation (baslerweb.com) Basler Camera Module out-of-box with i.MX 8M Plus Applications Processor. (Video: Basler Camera Module out-of-box with i.MX 8M Plus Applications Processor | NXP Semiconductors) Steps After setting up the hardware we will need to turn on the iMX8M Plus and follow these steps: 1. Stop the boot process on Uboot by pressing any key. 2. Use the following command to list interfaces. => mmc list Output example => FSL_SDHC: 1 (SD) => FSL_SDHC: 2 The above command will show you the device number in this example for SD, the device number is 1. 3. Then use fatls <interface> <device[:partition]> [<directory>] fatls mmc 1:1 (Device 1 : Partition 1) With this command, we will be able to list device tree files. => fatls mmc 1:1 4. Select imx8mp-evk-basler.dtb or imx8mp-evk-dual-basler.dtb and use the command editenv fdtfile.  => editenv fdtfile Output example edit: imx8mp-evk-basler.dtb 5. In edit command line put the selected device tree (*.dtb). 6. Use saveenv command to save environment and continue with the boot process. 7. Using the terminal and go to /opt/imx8-isp/bin and execute the script run.sh. $ ./run.sh -c basler_1080p60 -lm 8. Use the command gst-device-monitor-1.0 to list devices. Here you will find the path to the camera device. $ gst-device-monitor-1.0 Output example Device found: name : VIV class : Video/Source caps : video/x-raw, format=YUY2, width=[ 176, 4096, 16 ], height=[ 144, 3072, 8 ], pixel-aspect-ratio=1/1, framerate={ (fraction)30/1, (fraction)29/1, (fraction)28/1, (fraction)27/1, (fraction)26/1, (fraction)25/1, (fraction)24/1, (fraction)23/1, (fraction)22/1, (fraction)21/1, (fraction)20/1, (fraction)19/1, (fraction)18/1, (fraction)17/1, (fraction)16/1, (fraction)15/1, (fraction)14/1, (fraction)13/1, (fraction)12/1, (fraction)11/1, (fraction)10/1, (fraction)9/1, (fraction)8/1, (fraction)7/1, (fraction)6/1, (fraction)5/1, (fraction)4/1, (fraction)3/1, (fraction)2/1, (fraction)1/1 } ... properties: udev-probed = true device.bus_path = platform-vvcam-video.0 sysfs.path = /sys/devices/platform/vvcam-video.0/video4linux/video2 device.subsystem = video4linux device.product.name = VIV device.capabilities = :capture: device.api = v4l2 device.path = /dev/video2 v4l2.device.driver = viv_v4l2_device v4l2.device.card = VIV v4l2.device.bus_info = platform:viv0 v4l2.device.version = 393473 (0x00060101) v4l2.device.capabilities = 2216693761 (0x84201001) v4l2.device.device_caps = 69206017 (0x04200001) gst-launch-1.0 v4l2src device=/dev/video2 ! ... 9. Finally, use gstreamer to verify proper operation. (With this gstreamer pipeline you will see a new window with the camera output. Then, just rotate the lens to acquire the correct focus) $ gst-launch-1.0 -v v4l2src device=/dev/video2 ! "video/x-raw,format=YUY2,width=1920,height=1080" ! queue ! imxvideoconvert_g2d ! waylandsink Basic description of Gstreamer Pipeline gst-launch-1.0 -v: The option -v enables the verbose mode to get detailed information of process. v4l2src device=/dev/video2: Select input device in this case the camera is on path /dev/video3. "video/x-raw,format=YUY2,width=1920,height=1080": Received format from camera. queue: This command is a buffer between camera recording process and the following image process, this command help us to interface two process and prevent blocking where each process has different speeds, in other words, when a process A is faster than process B. imxvideoconvert_g2d: This proprietary plugin uses hardware acceleration to perform rotation, scaling, and color space conversion on video frames. waylandsink : This command creates its own window and renders the decoded frames processed previously. 10. Result     I hope this article will be helpful. Best regards, Brian.

Platform: i.MX8MP SW:Linux 5.4.70.2.3.0 On current linux BSP, PCIE driver does not support Hot-plug, customers wants to turn off PCIE device to save power, attached is guide. Remove PCIE device driver Suspend PCIE driver Turn off PCIE device power supply Turn on PCIE device power supply Resume PCIE driver Rescan PCIE device Load PCIE device driver

This is a summary for the software lockup issue found in the following platform: −i.MX8/8X −Linux 4.14.98_2.3.3   Issue description: •Issue happens during the boot procedure, at the systemd stage. •The symptom of the issue: −From user perspective, the symptom varies, but mainly fall into several types: §At the console, there may be login prompt, but no response (only echo) when input user/password. Unable to login. §Some user service in systemd failed to start. E.g. weston. −When checking the task status using sysrq (w/t), many tasks, including some kernel core tasks stays in “D” (uninterruptable sleep) state. E.g. agetty, login, chvt, etc. •Kernel itself is still alive. This can be verified by triggering some drivers, such as plugin a USB device. Issue can be reproduced on MEK through long time stress.   Please refer to the doc/patch attached for details.

i.MX8 series contains internal HiFi4 DSP. It is targeted for Audio related signal processing. SOF (Sound Open Firmware) is open source audio DSP firmware, driver and SDK. This document introduces basic theory about IIR/FIR digital filters, how to design IIR/FIR digital filters and the Equalizer filters implementation by SOF. After that, the document also describes how HiFi4 DSP MAC engine accelerate the EQ filters calculation.

Hello, here Jorge. On this post I will explain how to configure, record and play audio using an i.MX 8MIC-RPI-MX8 Board. Requirements: I.MX 8M Mini EVK Linux Binary Demo Files - i.MX 8MMini EVK (L5.15.52_2.1.0) i.MX 8MIC-RPI-MX8 Board Serial console emulator (Tera Term, Putty, etc.) Headphones/speakers The 8MIC-RPI-MX8 accessory board is designed for voice enabled application prototyping and development on the i.MX 8M family. The board plugs directly into the 40-pin expansion connector on the i.MX 8M Mini and Nano EVK’s. Some features about this board are: 8 PDM Microphones 8 monochrome LEDs 4 multi-color LEDs 2 status LEDs 4 pushbuttons Microphone Mute Switch Microphone geometry switch Connecting the i.MX 8MIC-RPI-MX8 Board. The i.MX 8MIC-RPI-MX8 Board has a 40-pin expansion connector that you can plug it directly to the EVK board. Ensure that pin 1 of the 8MIC-RPI-MX8 is aligned with pin 1 on the EVK J1001 as is showed on the next figure:  Selecting the device tree on the board. Once the pre-compiled image is flashed on the board (Flashing Linux BSP using UUU) and you connected the 8MIC-RPI-MX8 it is necessary to select the correct device tree to handle 8MIC board. On U-boot check the available .dtb files on the BSP using the next command: u-boot=> fatls mmc 2:1 And you will get the corresponding list of .dbt files:  On this case we are working with an I.MX 8M Mini EVK and the corresponding .dtb file is: imx8mm-evk-8mic-revE.dtb To select it you need to set the environment variable and save it with: u-boot=> setenv fdtfile imx8mm-evk-8mic-revE.dtb u-boot=> saveenv Doble check it using: u-boot=> printenv fdtfile   Now it is time to boot Linux using the next command: u-boot=> boot Recording audio with the i.MX 8MIC-RPI-MX8 Board. The Advanced Linux Sound Architecture (ALSA) provides audio and MIDI functionality to the Linux operating system. ALSA has the following significant features: Efficient support for all types of audio interfaces, from consumer sound cards to professional multichannel audio interfaces. Fully modularized sound drivers. SMP and thread-safe design. User space library (alsa-lib) to simplify application programming and provide higher level functionality. Support for the older Open Sound System (OSS) API, providing binary compatibility for most OSS programs. Once we are on Linux, we can check our audio codecs detected on the board using: arecord -l   Now, to record audio we need to use the ALSA arecord command to start recording with IMX8 boards, there are different options that you can check on the next link. On this case we are going to use the next: arecord -D hw:imxaudiomicfil -c8 -f s16_le -r48000 -d10 sample.wav -D: selects the device. -c: selects the number of channels on the recording. -f: selects the format. -r: selects the sample rate. -d: determinate the duration recording time in seconds. sample.wav: Is the name of the resulting audio file. Running the last command, we started to record audio. It is time to make some noise and record it!   Playing audio from IMX8 boards. Now it is time to connect our headphones or speakers to the jack.   Also, as on arecord command you can check the devices where you can play audio from the board using the next command: aplay -l And you will get all the codecs to play audio:   To play our recordings we need to use the ALSA aplay command, it is important to select the correct audio codec to hear the audio from the jack on the board: aplay -Dplughw:3,0 sample.wav -D: selects the device. sample.wav: Is the name of audio file to play   Hope this will helpful for people who wants to record audio using PDM microphones and playing audio from IMX8 boards. Best regards.

Symptoms   Bridge mode on EQoS module will not work since Linux Kernel 5.10_2.2.0. Platforms impacted: i.MX8MP/i.MX8DXL/i.MX93   Diagnosis   When eqos module(eth1) is added to the bridge using brctl, it will first set eth1 to promiscuous mode and then set the VLAN for this bridge with a filter VID value of 1. Before adding Intel's patch, there is no problem. c89f44ff10fd net: stmmac: Add support for VLAN promiscuous mode However, when Intel's patch sets up the filter, if it finds that the promiscuous mode is turned on, it will turn off the VLAN Tag function. And it adds a judgment on whether promiscuous mode has been turned on in the function of configuring VID. Returns an error if promiscuous mode is found. Because the patch has turned off the VLAN tag function when promiscuous mode is enabled, which conflicts with continuing to configure the VID. Workaround   This patch is okay for aarch64 platform to solve this issue. diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c index c25bfecb4a2d..2dc548b54b1c 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c @@ -481,12 +481,6 @@ static int dwmac4_add_hw_vlan_rx_fltr(struct net_device *dev, if (vid > 4095) return -EINVAL; - if (hw->promisc) { - netdev_err(dev, - "Adding VLAN in promisc mode not supported\n"); - return -EPERM; - } - /* Single Rx VLAN Filter */ if (hw->num_vlan == 1) { /* For single VLAN filter, VID 0 means VLAN promiscuous */ @@ -536,12 +530,6 @@ static int dwmac4_del_hw_vlan_rx_fltr(struct net_device *dev, { int i, ret = 0; - if (hw->promisc) { - netdev_err(dev, - "Deleting VLAN in promisc mode not supported\n"); - return -EPERM; - } - /* Single Rx VLAN Filter */ if (hw->num_vlan == 1) { if ((hw->vlan_filter[0] & GMAC_VLAN_TAG_VID) == vid) {  

This article describes how to integrate NXP WiFi & BT module into i.MX platform, some debug tips, how to test, etc. Although it takes i.MX8MM as example, it is also suitable for all i.MX8 serials platform.

This is a simple known-how for how to enable KASAN on L4.14.98 with i.MX8/8X and also a collection of related patches for fixing issues reported by KASAN.   Enable KASAN: It's very simple, just enable "CONFIG_KASAN" in kernel configuration. Besides this, adjusting the kernel's loading address is also required, due to KASAN (which will alloc more memory as a "tracker" for each allocation). For e.g., on imx8qxp MEK, we need to change the kernel loadaddr in uboot:     0x80280000 --> 0xE0280000 through uboot env: setenv loadaddr 0xE0280000 After this, it supposed to be working.   KASAN related patches on L4.14.98: KASAN will do detection/sanitizing for any memory allocation/access. In case of L4.14.98 on i.MX8/8X, there're several "BUG" reported by KASAN in default BSP. The attached patches are a collection for these issues.    Note: not all "BUG" reported by KASAN are really bug. Most of them are just some programming rule related problems and may not really cause memory access violation.

On behalf of Gopise Yuan. A simple sharing for enabling some kernel options for using “lockdep”. It will be useful for detecting some deadlock (spinlock/mutex/…) issue. Validated on L4.14.x, and, found a lock violation…

Sometimes we got CMA allocation error while there're still many free pages in CMA area.  This article analyze several cases/reason for this kind of failure.

BSP: L5.15.5_1.0.0   Platform: i.MX8MPlus EVK   1. Parameter preparation For more parameter calculation, please refer to: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/iMX-8M- Mini-Register-Programming-Aid-DRAM-PLL-setting/ta-p/111209  For 1866MHz LPDDR4, we need a DRAM PLL size of 933MHz. The PLL dividing parameters are: m=622,p=16,s=0, k=0.   2. Calibration and stress test with DDR Tool 2.1 Creating a test script for 1866MHz Here we copy the script from another file (e.g. 2000MHz) and modify the contents of the script.   2.2 Modify the script to adapt to 1866MHz 2.3 Download the test script After selecting the ddr script we created, click on the download button   2.4 Calibrating the stress test Set the core clock of the chip's cpu to 1.2GHz, then click the Calibration button to calibrate, then click Gen Code to generate the lpddr4_timing.c file. Set the start frequency to 1866MHz for the stress test.   2.5 Modify lpddr4_timing.c We need to modify the generated lpddr4_timing.c file to change the maximum speed to 3732MTS.   3. SPL patch After getting the correct lpddr4_timing.c file, the SPL code also needs to be modified to add support for the 933MHz DRAM PLL. diff --git a/arch/arm/mach-imx/imx8m/clock_imx8mm.c b/arch/arm/mach-imx/imx8m/clock_imx8mm.c index e39f238fdf...5622a6334e 100644 --- a/arch/arm/mach-imx/imx8m/clock_imx8mm.c +++ b/arch/arm/mach-imx/imx8m/clock_imx8mm.c @@ -55,6 +55,7 @@ static struct imx_int_pll_rate_table imx8mm_fracpll_tbl[] = { PLL_1443X_RATE(650000000U, 325, 3, 2, 0), PLL_1443X_RATE(600000000U, 300, 3, 2, 0), PLL_1443X_RATE(594000000U, 99, 1, 2, 0), + PLL_1443X_RATE(933000000U, 622, 16, 0, 0), PLL_1443X_RATE(400000000U, 400, 3, 3, 0), PLL_1443X_RATE(2660000U, 266, 3, 3, 0), PLL_1443X_RATE(167000000U, 334, 3, 4, 0), diff --git a/drivers/ddr/imx/imx8m/ddrphy_utils.c b/drivers/ddr/imx/imx8m/ddrphy_utils.c index 326b92d784..ebd005bc2b 100644 --- a/drivers/ddr/imx/imx8m/ddrphy_utils.c +++ b/drivers/ddr/imx/imx8m/ddrphy_utils.c @@ -117,6 +117,10 @@ void ddrphy_init_set_dfi_clk(unsigned int drate) dram_pll_init(MHZ(1000)); dram_disable_bypass(); break; + case 3732: + dram_pll_init(MHZ(933)); + dram_disable_bypass(); + break; case 3200: dram_pll_init(MHZ(800)); dram_disable_bypass();   4. Test results   Reference blog. DDR Tool: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX-8M-Family-DDR-Tool-Release/ta-p/1104467  RPA: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX-8MPlus-m865S-DDR-Register-Programming-Aids-RPA/ta-p/1235352 

This is simple known-how for how to implement "boot animation" with DRM under i.MX8/X + Linux:   Code to refer to: ========================================================================= 1. kmscube: Either open source one or the customized on for i.MX will be OK: https://cgit.freedesktop.org/mesa/kmscube/ https://source.codeaurora.org/external/imx/kmscube-imx/ 2. Android display HAL: KmsDisplay.cpp   Known-how: ========================================================================= 1. Only one application can grab the master role of the DRM device. If need to control DRM from two applicaiton simultanously, possible solution:     A, Use "controlD" node instead of "card" node in /dev/dri/. This requires L4.14 or before. This device node was removed by two commits in L4.14.x:           8a357d10043c75e980e7fcdb60d2b913491564af           6449b088dd51dd5aa6b38455888bbf538d21f2fc     Can be brought back by reverting these two commits in L4.14.98.     B, Use framebuffer emulator to emulate a FB device (/dev/fb0). (not recommended due to lack of vsync). 2. Some kernel functions will re-config the DRM device during boot. This will cause display abnormal after user application has configured the DRM device. Better to disable these kernel features:       CONFIG_DRM_FBDEV_EMULATION       CONFIG_FRAMEBUFFER_CONSOLE 3. Use atomic mode of KMS API instead of legacy mode for any dynamically screen drawing application, such as video, game and etc. Atomic mode will have much better performance compare to legacy mode. The kmscube has sample code for both mode. 4. Better to do commit checking before doing any real commit, especially when doing display during boot. Sometimes some internal component in DRM is not fully ready after card device is present.       DRM_MODE_ATOMIC_ALLOW_MODESET 5. If video playback will be used, some points to remind:     a, Sample code for direct video decoding (in unit-test): imx-test/test/mxc_v4l2_vpu_test/     b, VPU in i.MX8/X only support tiled NV12 output and it has pixel alignment requirement (128). Need to use CPU or G2D to do un-tile, CSC and cropping. Sample code: <android>/vendor/nxp/fsl_imx_omx/OpenMAXIL/src/component/v4l2_common/G2dProcess.cpp If using G2D under Linux, it will support un-tile directly (through OpenCL internally).

 This article instruct customer how to develop on i.MX8MP NPU and how to debug performance.