Platform: Demo images, i.MX8MPlus EVK   Some customer need test ffs gadget function on i.MX8MPlus EVK. Here is demo for ffs test, please connect EVK and Ubuntu PC before test.   Test script: #!/bin/sh # Setup the device (configfs) modprobe libcomposite mkdir -p config mount none config -t configfs cd config/usb_gadget/ mkdir g1 cd g1 echo 0x1fc9 >idVendor echo 0x0146 >idProduct mkdir strings/0x409 echo 12345 >strings/0x409/serialnumber echo "Signal 11" >strings/0x409/manufacturer echo "Test" >strings/0x409/product mkdir configs/c.1 mkdir configs/c.1/strings/0x409 echo "Config1" >configs/c.1/strings/0x409/configuration # Setup functionfs mkdir functions/ffs.usb0 ln -s functions/ffs.usb0 configs/c.1 cd ../../../ mkdir -p ffs mount usb0 ffs -t functionfs cd ffs ffs-test 64 & # from the Linux kernel, with mods! sleep 3 cd .. # Enable the USB device echo 38100000.usb > config/usb_gadget/g1/UDC   EVK log root@imx8mpevk:~# ./test2.sh [ 17.859597] file system registered ffs-test: dbg: ep0: writing descriptors (in v2 format) ffs-test: dbg: ep0: writing strings ffs-test: dbg: ep1: starting ffs-test: dbg: ep2: starting ffs-test: dbg: ep1: starts ffs-test: dbg: ep0: starts ffs-test: dbg: ep2: starts Event BIND Event ENABLE Ubuntu PC log: lzm@lzm-GL552VW:~$ lsusb -D /dev/bus/usb/001/008 Device: ID 1fc9:0146 NXP Semiconductors Test Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.10 bDeviceClass 0 bDeviceSubClass 0 bDeviceProtocol 0 bMaxPacketSize0 64 idVendor 0x1fc9 NXP Semiconductors idProduct 0x0146 bcdDevice 6.01 iManufacturer 1 Signal 11 iProduct 2 Test iSerial 3 12345 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 0x0020 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 4 Config1 bmAttributes 0x80 (Bus Powered) MaxPower 2mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 2 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 0 bInterfaceProtocol 0 iInterface 5 Source/Sink Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 2 Transfer Type Bulk Synch Type None Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x01 EP 1 OUT bmAttributes 2 Transfer Type Bulk Synch Type None Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 1 Binary Object Store Descriptor: bLength 5 bDescriptorType 15 wTotalLength 0x0016 bNumDeviceCaps 2 USB 2.0 Extension Device Capability: bLength 7 bDescriptorType 16 bDevCapabilityType 2 bmAttributes 0x0000010e BESL Link Power Management (LPM) Supported BESL value 256 us SuperSpeed USB Device Capability: bLength 10 bDescriptorType 16 bDevCapabilityType 3 bmAttributes 0x00 wSpeedsSupported 0x000f Device can operate at Low Speed (1Mbps) Device can operate at Full Speed (12Mbps) Device can operate at High Speed (480Mbps) Device can operate at SuperSpeed (5Gbps) bFunctionalitySupport 1 Lowest fully-functional device speed is Full Speed (12Mbps) bU1DevExitLat 0 micro seconds bU2DevExitLat 0 micro seconds Device Status: 0x0001 Self Powered  

1. Description: On the i.MX Android camera HAL, It only supports YUYV sensor, regardless of whether the sensor is connected to ISP or ISI. Some users want to customize the sensor format, such as UYVY or raw, they need a guide to do this, this document intends to describe how to implement raw camera sensor on i.MX8MP android, and output raw data. Note: Base on  i.MX 8M plus, Android12_1.0.0.  2. Camera HAL Android's camera hardware abstraction layer (HAL) connects the higher level camera framework APIs in android.hardware.camera2 to your underlying camera driver and hardware. For more detail information, please refer to AOSP document: https://source.android.google.cn/docs/core/camera/camera3_requests_hal?hl=en while on I.MX camera HAL, the camera subsystem can be divided into several parts: TonyLiao_suzhou_0-1669960499301.png   Camera framework:  frameworks\av\camera Camera service:    frameworks\av\services\camera\libcameraservice\ Camera provider: hardware/interfaces/camera/provider/ hardware/google/camera/common/hal/hidl_service/               hidl_service dlopen the camera HAL3. Camera HAL3:   vendor\nxp-opensource\imx\camera\ Camera driver:   vendor/nxp-opensource/kernel_imx/drivers/media/i2c  It's callstack can be list as follow:  TonyLiao_suzhou_1-1669960725413.png TonyLiao_suzhou_2-1669960738653.png There are 2 streams on pipeline, preview stream need 3 buffers and capture stream need 2 buffers: CameraDeviceSessionHwlImpl: ConfigurePipeline, stream 0: id 0, type 0, res 2592x1944, format 0x21, usage 0x3, space 0x8c20000, rot 0, is_phy 0, phy_id 0, size 8388608 CameraDeviceSessionHwlImpl: ConfigurePipeline create capture stream CameraDeviceSessionHwlImpl: ConfigurePipeline, stream 1: id 1, type 0, res 1024x768, format 0x22, usage 0x100, space 0x0, rot 0, is_phy 0, phy_id 0, size 0 CameraDeviceSessionHwlImpl: ConfigurePipeline create preview stream You can use this command to dump the stream input/output: "setprop vendor.rw.camera.test 1" to dump steam 0. "setprop vendor.rw.camera.test 2" to dump steam 1. Before you implement the command, you need to run “su; setenforce 0” to close the SeLinux, the data is dumped as "/data/x-src.data", "/data/x-dst.data", where "x" is the stream id as "0, "1,", "2", ...   Preview Stream Capture Stream ID 1 0 Resolution 1024*768 2592*1944 Format HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED (34) HAL_PIXEL_FORMAT_BLOB (33) Usage 0x900 GRALLOC_USAGE_HW_TEXTURE      (0x100) GRALLOC_USAGE_HW_COMPOSER  (0x800) 0x3 GRALLOC_USAGE_SW_READ_OFTEN Data space 0 HAL_DATASPACE_V0_JFIF  The following usage will be added by framework to distinguish preview and video:GRALLOC_USAGE_HW_VIDEO_ENCODER 3. Raw support  3.1 system modification  The default image for i.MX 8M Plus EVK supports basler + basler and the cameras can work after the image is flashed and boot up, it’s camera with ISP, but we need ISI to process raw. You should refer to Android_User’s_Guide.pdf, you need find the correct version, as Android12_2.0.0 is different with Android12_1.0.0. To make cameras work with Non-default images, execute the following additional commands: Only OV5640 (CSI1) on host： TonyLiao_suzhou_3-1669960978453.png As we use OV2775 which support 1920*1080, unpacked raw12, the json file: TonyLiao_suzhou_4-1669961020578.png 3.2 DTB modification  1. Firstly, change the BoardConfig.mk to generate dtbo-imx8mp-ov2775.img device\nxp\imx8m\evk_8mp\BoardConfig.mk: TARGET_BOARD_DTS_CONFIG += imx8mp-ov2775:imx8mp-evk-ov2775.dtb 2.  Secondly, add imx8mp-evk-ov2775.dts to vendor\nxp-opensource\kernel_imx\arch\arm64\boot\dts\freescale 3. Change imx8mp-evk-ov2775.dts, connect OV2775 to ISI: &isi_0 { status = "okay"; }; &isp_0 { status = "disabled"; }; &dewarp { status = "disabled"; }; 4. Build dtbo image and flash it to board: ./imx-make.sh dtboimage -j4 fastboot flash dtbo dtbo.img  3.3 Sensor driver modification  I use the OV2775 driver from the isp side, be careful that all the function such as g_frame_interval and enum_frame_size should be implemented, or the HAL will get wrong parameters and return error. static struct v4l2_subdev_video_ops ov2775_subdev_video_ops = { .g_frame_interval = ov2775_g_frame_interval, .s_frame_interval = ov2775_s_frame_interval, .s_stream = ov2775_s_stream, }; static const struct v4l2_subdev_pad_ops ov2775_subdev_pad_ops = { .enum_mbus_code = ov2775_enum_mbus_code, .set_fmt = ov2775_set_fmt, .get_fmt = ov2775_get_fmt, .enum_frame_size = ov2775_enum_frame_size, .enum_frame_interval = ov2775_enum_frame_interval, };  3.4 ISI driver modification  We need to add raw format on ISI driver: }, { .name = "RAW12 (SBGGR12)", .fourcc = V4L2_PIX_FMT_SBGGR12, .depth = { 16 }, .color = MXC_ISI_OUT_FMT_RAW12, .memplanes = 1, .colplanes = 1, .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, }, { .name = "RAW10 (SGRBG10)", .fourcc = V4L2_PIX_FMT_SGRBG10, .depth = { 16 }, .color = MXC_ISI_OUT_FMT_RAW10, .memplanes = 1, .colplanes = 1, .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, } 3.5 Camera HAL modification As there are preview stream and capture stream on the pipeline. GPU does not support raw format, it will print error log when application set raw format:    02-10 18:49:02.162 436 436 E NxpAllocatorHal: convertToMemDescriptor Unsupported fomat PixelFormat::RAW10 02-10 18:49:02.163 2390 2445 E GraphicBufferAllocator: Failed to allocate (1920 x 1080) layerCount 1 format 37 usage 20303: 7 02-10 18:49:02.163 2390 2445 E BufferQueueProducer: [ImageReader-1920x1080f25m2-2390-0](id:95600000002,api:4,p:2148,c:2390) dequeueBuffer: createGraphicBuffer failed 02-10 18:49:02.163 2390 2405 E BufferQueueProducer: [ImageReader-1920x1080f25m2-2390-0](id:95600000002,api:4,p:2148,c:2390) requestBuffer: slot 0 is not owned by the producer (state = FREE) 02-10 18:49:02.163 2148 2423 E Surface : dequeueBuffer: IGraphicBufferProducer::requestBuffer failed: -22 02-10 18:49:02.163 2390 2445 E BufferQueueProducer: [ImageReader-1920x1080f25m2-2390-0](id:95600000002,api:4,p:2148,c:2390) cancelBuffer: slot 0 is not owned by the producer (state = FREE) 02-10 18:49:02.164 2148 2423 E Camera3-OutputStream: getBufferLockedCommon: Stream 1: Can't dequeue next output buffer: Invalid argument (-22)   Modifying the gpu code is not recommended. When preview stream, it's pixel format is fixed to HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED, it needs YUYV format, in this patch, we don't convert raw12 to yuyv, just copy the buffer from input to output, so the preview stream is raw12 actually. When capture stream, we use the Blob format, which usually used for JPEG format. when we find the format is Blob pass down by application, camera HAL will copy the buffer from input to output directly. You can check the detail on function ProcessCapturedBuffer(),    4. Application and Tool 4.1 Application  The test application on the attachment “android-Camera2Basic-master_application.7z”, It's basically a common camera application, it set the capture stream format to blob:  Size largest = Collections.max( Arrays.asList(map.getOutputSizes(ImageFormat.JPEG)), new CompareSizesByArea()); mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(), ImageFormat.JPEG, /*maxImages*/2); 4.2 Tool We use 7yuv tool to check the raw12 format, which is captured by applicable or dump by HAL, you need set the parameter on the right side:   TonyLiao_suzhou_5-1669966390009.png ImageJ tool also can be used to review raw format.

  1.overwrite the sources/meta-freescale/recipes-security/optee-imx with optee-imx.zip 2.add below code to conf/local.conf DISTRO_FEATURES_append += " systemd" DISTRO_FEATURES_BACKFILL_CONSIDERED += "sysvinit" VIRTUAL-RUNTIME_init_manager = "systemd" VIRTUAL-RUNTIME_initscripts = "systemd-compat-units" MACHINE_FEATURES_append += "optee" DISTRO_FEATURES_append += "optee" IMAGE_INSTALL_append += "optee-test optee-os optee-client optee-examples" 3.bitbake optee-examples or bitbake imx-image-xxx You can directly install optee-examples_3.11.0-r0_arm64.deb in your device.  

1.  Introduction 1.1.        Purpose This application note introduces a procedure of how to port AVB/TSN stack and run referring feature demos on i.MX8DXL board. This can help users who want to run AVB/TSN demos to quickly understand and customized their own codes. Since many of the standards are only for TSN switch/bridges and i.MX8DXL is design to be a TSN/AVB endpoint, the demos did not implement a full stack or full standards. They only demonstrated the basic end-to-end point (talker to listener) A/V streaming without bridge or switch. The software used for example in this documentation are based on the opensource such as gstreamer and alsa utils.   1.2.        Overview 1.2.1.     AVB/TSN AVB (Audio Video Bridging) is a common name for the set of technical standards which provide improved synchronization, low-latency, and reliability for switched ethernet network. AVB was initially developed by the IEEE Audio Video Bridging task group of the IEEE 802.1 standards committee. In November 2012, AVB group was renamed to TSN (Time-Sensitive Networking) task group to reflect the expanded scope of its work, which is to provide the specifications that will allow time-synchronized low latency streaming services through IEEE 802 networks. The referring standards shows as follows: Peter_Liu_0-1622613902450.png     TSN protocol additions QoS components supported in HW TSN MAC + SW driver Managed Object components expose i/f to allow support of standardized network config protocols (local & remote) Transport API to allow other transport layer to use TSN QoS Stack extensions to map traffic priority to application task scheduling Real Time, gPTP based, Best Effort 1.2.2.     Demo introduction Peter_Liu_1-1622613930890.png   The two streams are defined as below to grantee time sensitive (sub-microsecond synchronization), low latency and bandwidth on the ethernet: Stream A: SR class A, AVTP Audio Format, PCM 16-bit sample, 48 kHz, stereo, 12 frames per AVTPDU. Stream B: SR class B, AVTP Compressed Video Format, H.264 profile High, 1920x1080, 30 fps. The two TSN streams would be allocated into different TC (traffic control) class for egress. Different TC class would be mapped to different hardware queues with specific DMA channel which supported by ENET_OoS IP. The demos were built by follow blocks: Peter_Liu_2-1622613954064.png   Linux Traffic Control: streams egress control Linux ptp: clock sync in network Libavtp: Time Sensitive Applications AV Transport protocol Gstreamer: avtp plugin uses the libavtp to transmit and receive AVTP audio/video (audio pcm, video h264).   1.2.3.     Traffic control Multiply queue qdiscs + CBS: The CBS class is actually handled by hardware IP to select which queue for transmitting. Peter_Liu_3-1622613973901.png   CBS parameters come straight from the IEEE 802.1Q-2018 specification. They are the following: idleSlope: rate credits are accumulated when queue isn’t transmitting; sendSlope: rate credits are spent when queue is transmitting; hiCredit: maximum amount of credits the queue is allowed to have; loCredit: minimum amount of credits the queue is allowed to have; Peter_Liu_4-1622613992866.png     2.  Build demo 2.1.        Build yocto $ DISTRO=fsl-imx-xwayland MACHINE=imx8dxlevk source imx-setup-release.sh -b ./xwayland $ bitbake imx-image-full Prepare a SD card and burn it with the built out images. 2.2.        Rebuild kernel Rebuild the kernel after applying the 0001-qenet-add-queue-avoid-panic.patch, and overwrite the Image and imx8dxl-evk.dtb on the boot partition of the SD card. 2.3.        Install the toolchain $ bitbake -f fsl-image-validation-imx -c populate_sdk $ sh tmp/deploy/sdk/fsl-imx-xwayland-glibc-x86_64-fsl-image-validation-imx-aarch64-imx8dxlevk-toolchain-5.4-zeus.sh The toolchain would be installed into /opt/fsl-imx-xwayland/5.4-zeus   2.4.        Create a install folder $ mkdir <your install folder> Create a folder to install all of the shared libraries, binaries and configure files which built out manually in this doc. After built done, you should copy all of the contents in this folder to target board root.   2.5.        Build libavtp $ source /opt/fsl-imx-xwayland/5.4-zeus/environment-setup-aarch64-poky-linux $ git clone https://github.com/Avnu/libavtp.git $ cd libavtp $ meson build --prefix=<your install folder>/usr $ ninja -C build Copy the built out .so and .pc into the toolchain rootfs: $ sudo cp build/libavtp.so* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/lib $ sudo cp build/meson-private/*.pc /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/lib/pkgconfig/ Copy the .so into the install folder: $ cp build/libavtp.so* <install folder>/usr/lib/ To make sure you have avtp package installed correctly:     $ pkg-config --list-all | grep avtp   2.6.        Build ALSA aaf plugin $ cd <yocto build>/tmp/work/aarch64-poky-linux/alsa-plugins/1.1.9-r0/alsa-plugins-1.1.9 $ ./configure --build=x86_64-linux --host=aarch64-poky-linux --target=aarch64-poky-linux --prefix=<install folder>/usr --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=<yocto build>/xwayland/tmp/work/aarch64-poky-linux/alsa-plugins/1.1.9-r0/recipe-sysroot --disable-static --enable-aaf --disable-jack --disable-libav --disable-maemo-plugin --disable-maemo-resource-manager --enable-pulseaudio --enable-samplerate --with-speex=lib $ make $ make install   2.7.        Build Gstreamer AVTP plugins （1.17.x） 2.7.1.     Build Gstreamer core $ git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git $ patch -p1 < gstreamer-1.0-pass-build.patch $ meson build --prefix=<install folder>/usr $ ninja -C build $ sudo ninja -C build install After Gstreamer is installed into <your install folder>, please fix the “prefix” path in the .pc files by, and copy to the toolchain folders: $ cd <your install folder> $ grep -lR <your install folder> ./lib/pkgconfig/ | xargs sed -i 's/<your install folder>/\/usr/g' $ cp -rf ./usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/ 2.7.2.     Build gst-plugins-base $ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-base.git $ cd gst-plugins-base $ patch -p1 < gst-plugins-base-pass-build.patch $ meson build --prefix=<your install folder>/usr $ ninja -C build $ sudo ninja -C build install   2.7.3.     Build gst-plugins-bad $ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad.git $ cd gst-plugins-bad $ meson build --prefix=<your install folder>/usr $ ninja -C build $ sudo ninja -C build install   After gst-plugins-base and gst-plugins-bad installed into <your install folder>, please fix the “prefix” path in the .pc files and copy them into the toolchain folders: $ cd <your install folder> $ grep -lR <your install folder> ./lib/pkgconfig/ | xargs sed -i 's/<your install folder>/\/usr/g' $ cp -rf ./usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/   2.8.        Build H.264 SW plugins 2.8.1.     Build x264 As the yocto actually has the x264 recipes, but not included in our bblayers, we need to copy the x264 source into our bblayers path under <yocto>/source to build: $ cp -rf ./poky/meta/recipes-multimedia/x264 ./meta-openembedded/meta-multimedia/recipes-multimedia/ $ vi ./meta-openembedded/meta-multimedia/recipes-multimedia/x264_git.bb Remove the LICENSE_FLAGS line $ bitbake -f x264 -c do_install $ sudo cp -rf tmp/work/aarch64-poky-linux/x264/r2917+gitAUTOINC+72db437770-r0/image/usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/ 2.8.2.     Build gst-plugins-ugly $ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-ugly.git $ cd gst-plugins-ugly $ meson build --prefix=<your install folder>/usr $ ninja -C build $ sudo ninja -C build install   2.8.3.     Build libav $ cp -rf poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav meta-openembedded/meta-multimedia/recipes-multimedia/gstreamer-1.0/ Remove the LICENSE_FLAGS line $ vim ./poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.16.2.bb $ bitbake -f gstreamer1.0-libav -c do_install $ cp /opt/samba/nxf39444/imx-yocto-bsp-i.mx8dxl/xwayland/tmp/work/aarch64-poky-linux/gstreamer1.0-libav/1.16.2-r0/image/usr/lib/gstreamer-1.0/libgstlibav.so <your install folder>/usr/lib/gstreamer-1.0   2.8.4.     Install binaries Final step is to copy all of your built out files from <your install folder> into your board / root, and boot up the board. $ export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/ $ gst-inspect-1.0 To check if the above Gstreamer plugins we built out can be found by gst-instpect.   3.  System Setup 3.1.        VLAN The ENTE_QoS is assigned to eth0 instance. So create eth0.5 for vlan id 5: $ ip link add link eth0 name eth0.5 type vlan id 5 egress-qos-map 2:2 3:3 $ ip link set eth0.5 up   3.2.        Qdiscs The TSN control plane is implemented through the TC (Traffic Control) system. The transmission algorithms specified in the FQTSS (Forwarding and Queuing for Time-Sensitive Streams) chapter of IEEE 802.1Q-2018 are supported via TC Qdiscs (Queuing Discipline). 3.2.1.     MQPRIO qdisc $ tc qdisc add dev eth0 parent root handle 100 mqprio num_tc 3 map 0 0 2 1 0 0 0 0 0 0 0 0 0 0 0 0 queues 1@0 1@1 1@2 hw 1 3.2.2.     CBS qdisc Q1 CBS for audio, Q2 CBS for video: $ tc qdisc replace dev eth0 parent 100:3 handle 888 cbs idleslope 3648 sendslope -996352 hicredit 12 locredit -113 offload 1 $ tc qdisc replace dev eth0 parent 100:2 handle 777 cbs idleslope 98688 sendslope -901312 hicredit 153 locredit -1389 offload 1 3.2.3.     TimeSync Run the ptp4l and phc2sys in background, and use check_clocks to check the ptp sync works. $ ptp4l -i eth0 -f ./gPTP.cfg --step_threshold=1 & $ pmc -u -b 0 -t 1 "SET GRANDMASTER_SETTINGS_NP clockClass 248 clockAccuracy 0xfe offsetScaledLogVariance 0xffff currentUtcOffset 37 leap61 0 leap59 0 currentUtcOffsetValid 1 ptpTimescale 1 timeTraceable 1 frequencyTraceable 0 timeSource 0xa0" $ ./check_clocks -d eth0 4.  Run demo 4.1.        ALSA AAF audio To run the alsa AAF demo, please add aaf0 and converter0 plugin device into /etc/asound.conf: pcm.aaf0 {    type aaf    ifname eth0.5    addr 01:AA:AA:AA:AA:AA    prio 2    streamid AA:BB:CC:DD:EE:FF:000B    mtt 50000    time_uncertainty 1000    frames_per_pdu 12    ptime_tolerance 100 } pcm.converter0 {    type linear    slave {                  pcm "hw:0,0"                  format S16_LE    } } The “aaf0” plugin device defines the ethernet interface which AAF runs on, the socket priority which mapping to Traffic Class in kernel TC, the stream-id for the aaf streaming. The “converter0” plugin device is used for convert the S16_BE format to S16_LE for the wm8960 PCM audio.   Select one device as AVB talker, and run: $ speaker-test -p 25000 -F S16_BE -c 2 -r 48000 -D aaf0   Select one device as AVB listener, and run: $ arecord -F 25000 -t raw -f S16_BE -c 2 -r 48000 -D aaf0 | aplay -F 25000 -t raw -f S16_BE -c 2 -r 48000 -D converter0   You can hear the sound on the listener device.   You can also check which qdisc queue is used for AAF by: $ tc -s qdisc Peter_Liu_5-1622614072551.png   4.2.        Gstreamer AAF audio Select one device as AVB talker, and run: $ gst-launch-1.0 clockselect. \( clock-id=realtime audiotestsrc samplesperbuffer=12 is-live=true ! audio/x-raw,format=S16BE,channels=2,rate=48000 ! avtpaafpay mtt=50000000 tu=1000000 streamid=0xAABBCCDDEEFF000B processing-deadline=0 ! avtpsink ifname=eth0.5 address=06:98:c0:22:df:35 priority=3 processing-deadline=0 \)   Select one device as AVB listener, and run: $ gst-launch-1.0 clockselect. \( clock-id=realtime avtpsrc ifname=eth0.5 ! avtpaafdepay streamid=0xAABBCCDDEEFF000B ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=0 ! audioconvert ! audioresample !  alsasink device="hw:0,0" \)   5.  Packet sniffer Use tcpdump on board to dump the L2 ethernet packet: $ tcpdump -i eth0 ether proto 0x22f0 -w dump.pcap The AVTP ether protocol code is 0x22f0 embedded inside the ether frame, or you can use "vlan 5" VLAN id for tcpdump parameters to dump. Then open this dump.pcap in the windows/Linux PC by the wireshark tool, it will automatically show the protocol inside the package, it can also parser the IEEE1722 (AVTP) CVF/AFF package header as below: Peter_Liu_6-1622614099645.png   To measure the package latency from transmit port (talker) to receive port (listener), you can use the tcpdump on both end-points. And compare the Epoch Time the packet dumped: "Epoch Time: 1596252905.688243000 seconds". The delta of the epoch time of the same packet is around 100us~500us. This latency actually includes the AF_PACKET clone cost in kernel netfilter, also the tcpdump application schedule latency.   6.  Revision history summarizes the changes done to this document since the initial release. Table2. Revision history Revision number Date Substantive changes 1 5/2021 Initial release    

This is sample code for CSC settings for /dev/fb1. It can calculating the CSC matrix and updates in real time from given parameters(Brightness,Contrast,Saturation,Hue, and gamma).

Development environment: i.MX6Q SabreSD w/ L3.0.35_1.1.0_121218 release. Quoting from Wikipedia about exFAT (exFAT - Wikipedia, the free encyclopedia) as following: exFAT (Extended File Allocation Table) is a Microsoft file system optimized for flash drives. [3] It is proprietary and patent-pending. [1] It is supported in Windows XP and Windows Server 2003 with update KB955704, [2] Windows Embedded CE 6.0, Windows Vista with Service Pack 1, [4] Windows Server 2008, [5] Windows 7, Windows 8, Windows Server 2008 R2 (except Windows Server 2008 Server Core), Mac OS X Snow Leopard starting from 10.6.5, [6] Mac OS X Lion and OS X Mountain Lion. The history of support exFAT in Linux was since from 2.6.x, it involves several parts that will be described followingly. Part 1: Linux Kernel            Enable FUSE (Filesystem in userspace) feature in Kernel Config, then build a new uImage and module if set it to be "M"; Part 2: fuse-2.9.2.tar.gz            Download fuse-2.9.2.tar.gz from http://sourceforge.net/projects/fuse/files/fuse-2.X/, untar it, then build it with following commands: ./configure --prefix=/home/alanz/i.MX6_L3.0.35_121218/ltib/rootfs/usr --host=`/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-gcc -dumpmachine` --enable-lib --enable-util --enable-example --exec-prefix=/home/alanz/i.MX6_L3.0.35_121218/ltib/rootfs/usr make sudo make install Part 3: exfat-utils-1.0.1.tar.gz            Download exfat-utils-1.0.1.tar.gz from http://code.google.com/p/exfat/downloads/list, untar it, then build it with following command: sudo scons SYSROOT=/home/alanz/i.MX6_L3.0.35_121218/ltib/rootfs DESTDIR=/home/alanz/i.MX6_L3.0.35_121218/ltib/rootfs/sbin CC=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-gcc AR=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-ar RANLIB=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-ranlib STRIP=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-strip install Part 4: fuse-exfat.git git clone git://sources.progress-linux.org/git/releases/baureo-backports/packages/fuse-exfat.git  fuse-exfat.git cd fuse-exfat.git Replace the root SConstruct with the attached one, then execute command: sudo scons SYSROOT=/home/alanz/i.MX6_L3.0.35_121218/ltib/rootfs DESTDIR=/home/alanz/i.MX6_L3.0.35_121218/ltib/rootfs/sbin CC=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-gcc AR=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-ar RANLIB=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-ranlib STRIP=/opt/freescale/usr/local/gcc-4.6.2-glibc-2.13-linaro-multilib-2011.12/fsl-linaro-toolchain/bin/arm-none-linux-gnueabi-strip install After all above steps done, you can check whether the necessary files under your rootfs like I did as following: http://en.wikipedia.org/wiki/ExFAT#cite_note-uspatent-2alanz@alanz-VirtualBox:~/i.MX6_L3.0.35_121218/ltib$ find ./rootfs/ -name *exfat*./rootfs/sbin/exfatlabel ./rootfs/sbin/mount.exfat ./rootfs/sbin/fsck.exfat ./rootfs/sbin/dumpexfat ./rootfs/sbin/exfatfsck ./rootfs/sbin/mount.exfat-fuse ./rootfs/sbin/mkexfatfs ./rootfs/sbin/mkfs.exfat alanz@alanz-VirtualBox:~/i.MX6_L3.0.35_121218/ltib$ find ./rootfs/ -name *fuse*./rootfs/usr/include/fuse ./rootfs/usr/include/fuse/fuse_common_compat.h ./rootfs/usr/include/fuse/fuse_compat.h ./rootfs/usr/include/fuse/fuse_lowlevel_compat.h ./rootfs/usr/include/fuse/fuse_opt.h ./rootfs/usr/include/fuse/fuse_lowlevel.h ./rootfs/usr/include/fuse/fuse.h ./rootfs/usr/include/fuse/fuse_common.h ./rootfs/usr/include/fuse.h ./rootfs/usr/src/linux/include/linux/fuse.h ./rootfs/usr/lib/libfuse.so ./rootfs/usr/lib/libfuse.a ./rootfs/usr/lib/libfuse.so.2.9.2 ./rootfs/usr/lib/libfuse.la ./rootfs/usr/lib/libfuse.so.2 ./rootfs/sbin/mount.exfat-fuse Check Steps can be referenced by the steps presented on internet. NOTE: The directory name "/home/alanz/i.MX..." should be revised per your self development environment.

In our document there is about how to fuse in the u-boot, as follows you can see: Here we can use the mfgtool and these command to download the fuse to u-boot. 1/ Set the BOOT_MODE[1:0] to 00 2/Use the mfgtool to download the u-boot to RAM Use the mfgtool to download only the u-boot, so you have to annotate the code not about u-boot. Only u-boot download code left. As follows:  …………    Loading uboot.  Jumping to OS image. 3/When the u-boot boot up, print and go to the u-boot command line. U-Boot contains a tool, imxotp, which is used for fusing. The commands imxotp read addr and imxotp blow --force addr value read the data of Efuse. The addr is the register address of eFUSE, and the base address of eFUSE is 0x021BC0000, details you can refer to the section 46 of the iMX6DQRM.pdf p4016. And the The exact configuration please refer to the section 5 of the iMX6DQRM.pdf p315. Take the Sabrelite as an example the value of burning : imxotp blow --force 0x5 0x18000030 imxotp blow --force 0x6 0x10 Hope this can do some hope for you. About the efuse you also can refer to : https://community.nxp.com/thread/316232 

        The document will introduce how to setup cross‐compiling environment for android android7.1.1 BSP on Ubuntu 16.04.2 LTS, The purpose is to help i.MX customers create android BSP environment quickly, from this, save customer’s time and let them focus on the development of their product. Customer can compile android7.1.1 BSP according to the following steps: ‐‐Installing Ubuntu160.4.2 LTS 1. Running software updater to update system       Customer can download ubuntu‐16.04.2‐desktop‐amd64.iso from https://www.ubuntu.com/download/desktop Then install it to VMware workstation player v12 or PC, after finishing installation, use “Software Updater” to update system. 2. Installing necessary packages    Before compiling android7.1.1 source code, we need to install some neccesary software packages, see following, please! $ sudo apt-get install gnupg $ sudo apt-get install flex $ sudo apt-get install bison $ sudo apt-get install gperf $ sudo apt-get install build-essential $ sudo apt-get install zip $ sudo apt-get install zlib1g-dev $ sudo apt-get install libc6-dev $ sudo apt-get install lib32ncurses5-dev $ sudo apt-get install x11proto-core-dev $ sudo apt-get install libx11-dev $ sudo apt-get install lib32z1-dev $ sudo apt-get install libgl1-mesa-dev $ sudo apt-get install tofrodos $ sudo apt-get install python-markdown $ sudo apt-get install libxml2-utils $ sudo apt-get install xsltproc $ sudo apt-get install uuid-dev:i386 liblzo2-dev:i386 $ sudo apt-get install gcc-multilib g++-multilib $ sudo apt-get install subversion $ sudo apt-get install openssh-server openssh-client $ sudo apt-get install uuid uuid-dev $ sudo apt-get install zlib1g-dev liblz-dev $ sudo apt-get install liblzo2-2 liblzo2-dev $ sudo apt-get install lzop $ sudo apt-get install git-core curl $ sudo apt-get install u-boot-tools $ sudo apt-get install mtd-utils $ sudo apt-get install android-tools-fsutils $ sudo apt-get install openjdk-8-jdk 3. Downloading android7.1.1 source code, u‐boot, linux kernel 3.1 Downloading android7.1.1 source code 3.1.1 Getting source code from google .    if users can access google site, she can get source code accroding to steps in "Android_User's_Guide.pdf" released by NXP 3.1.2 Getting source code from the server of tsinghua university( this is for customer in China ) Steps: (1) Getting repo # cd ~ # mkdir myandroid # mkdir bin # cd bin # git clone https://aosp.tuna.tsinghua.edu.cn/android/git-repo.git/ # cd git‐repo # cp ./repo ../ (2) Modifying repo File Open ~/bin/repo file with 'gedit' and Change google address From REPO_URL = 'https://gerrit.googlesource.com/git-repo' To REPO_URL = ' https://gerrit-google.tuna.tsinghua.edu.cn/git-repo (3) Setting email address # cd ~/myandroid # git config --global user.email "email address" # git config --global user.name "name" [ Email & Name should be yours] (4) Modifying manifest.xml # ~/bin/repo init -u https://aosp.tuna.tsinghua.edu.cn/android/platform/manifest -b android-7.1.1_r13 # cd ~/myandroid/.repo # gedit manifest.xml Then change the value of fetch to " https://aosp.tuna.tsinghua.edu.cn/android/ ", like following: <manifest> <remote name="aosp" fetch="https://aosp.tuna.tsinghua.edu.cn/android/" /> <default revision="refs/tags/android-5.1.1_r1" ...... (5) # ~/bin/repo sync [Note] During runing repo sync, maybe errors will occur like the following: ...... * [new tag] studio‐1.4 ‐> studio‐1.4 error: Exited sync due to fetch errors Then 'repo sync' exits. But don't worry about it, continue to run the command please ! " ~/bin/repo sync", downloading source code will be continous. 3.2 Getting uboot source code $ cd ~/myandroid/bootable $ mkdir bootloader $ cd bootloader $ git clone git://git.freescale.com/imx/uboot-imx.git uboot-imx $ cd uboot-imx $ git checkout n7.1.1_1.0.0-ga 3.3 Downloading linux kernel $ cd ~/myandroid $ git clone git://git.freescale.com/imx/linux-imx.git kernel_imx $ cd kernel_imx $ git checkout n7.1.1_1.0.0-ga 4. Downloading android7.1.1 BSP source code and patch it above source code 4.1 Android7.1.1 BSP can be downloaded from the link: Android OS for i.MX Applications Processors|NXP  ---Board Support Packages (66) After downloading it, copy it to /opt/ 4.2 Patch it to android source code $ cd ~/myandroid $ source /opt/android_N7.1.1_1.0.0_source/code/N7.1.1_1.0.0/and_patch.sh $ help $ c_patch /opt/android_N7.1.1_1.0.0_source/code/N7.1.1_1.0.0/ imx_N7.1.1_1.0.0 If everything is OK, "c_patch" generates the following output to indicate the successful patch: 5. Compiling android7.1.1 BSP source code for i.MX boards $ export ARCH=arm $ export CROSS_COMPILE=~/myandroid/prebuilts/gcc/linux-x86/arm/armlinux-androideabi-4.9/bin/arm-linux-androideabi- $ cd ~/myandroid $ source build/envsetup.sh $ lunch sabresd_6dq-user $ make –j2   (Using 2 CPU cores to compile) Probably, users will enconter this error during compiling: [Solve it like this:] $ export ANDROID_JACK_VM_ARGS="-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx4g" $  cd ~/myandroid $ ./prebuilts/sdk/tools/jack-admin kill-server then run "make " to continue compiling. After compiling, we can see images in output path: -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Hope above items can help you! If customers have questions about the document, she can submit case to me by our saleforce system. the following is how to submit cases to us: ******************************************************************************************************************************************* In case a new customer is asking how to submit a technical case on nxp.com ,here is a template for your reference. 1) Please visit www.nxp.com and click on Support on the top of the webpage. 2) Select Sales and Support under Support Resources session. 3) Scroll down to the bottom ,click on “hardware & Software” . 4) Register by your business email to enter NXP Community 5) Get verification email and verify your account. 6) Select "contact support" on the top and click “submit a new case” to start the process. ******************************************************************************************** Then label : "please forward it to TIC Weidong Sun" in your content , I can get it. NXP TIC team Weidong.Sun 2017-03-16 in Shanghai China

MX6UL_Development_database_2017.4.21_V7.doc

This PDF is training material for showing examples on video encoding, video decoding, video streaming on an i.MX53QSB board.

       The document will introduce all steps for poring BCM4330/BCM43362 WIFI module to freescale android4.2.2 BSP, it includes these contents: --Hardware & Software Environment --Hardware Design Based on BCM43362 module --i.MX6 BSP configuration for WIFI module --BCM4330/BCM43362 dirver for linux 3.0.35 --Integrated to Android4.2.2    If customer has some questions with the porting, contact me , please ! my email address: [email protected] Freescale TICS team Weidong.sun 2015-08-20

Wireless HW module on i.MX 6 DQ HDMI dongle board is bcm4330 that is SDIO interface. Modprobe  default configuration will only insmod bcm4330.ko without any kernel module parameter, while bcm4330,ko needs extra firmware binary and nvram configuration file absolute path/filename  as parameter like firmware_path=/lib/firmware/bcm4330/fw_bcm4330.bin nvram_path=/lib/firmware/bcm4330/nvram_bcm4330.txt. To auto insmod bcm4330 kernel module with those parameters by modprobe we need a modprobe configuration file. Now create this file at /etc/modprobe.d/bc4330.conf, it's content as below: #For BCM4330 special install requirement options bcm4330 firmware_path=/lib/firmware/bcm4330/fw_bcm4330.bin nvram_path=/lib/firmware/bcm4330/nvram_bcm4330.txt Of course we need copy correct firmware and nvram configuration file to directory as /etc/modprobe.d/bc4330.conf set.

1. MediaPlayer Architecture 1.png   2.png   MediaPlayer is the server, MediaPlayerService and MediaPlayerService :: Client is the client. MediaPlayerService realize the IMediaPlayerService, the main function is to create the right player through the url sent by MediaPlayer::setDataSource MediaPlayerService :: Client realize the IMediaPlayer, the main function is to call the player created by MediaPlayerService to do those specific start, stop, resume, pause…      Entering NuplayerDriver means entering Android MultiMedia Framework.   2. NuPlayer   The playback video is mainly completed through Nuplayer. The figure below includes parser, decoder and render. 3.png   NuPlayer::Source is the parser module. Its interface looks like a combination of MediaExtractor and MediaSource NuPlayer::Decoder connects to CCodec for decoding. CCodec has a state pattern and pass MediaBuffers around with messages. NuPlayer::Render is responsible for rendering audio and also controls when to post video buffers back to NativeWindow for A/V sync. 3. Decoding Framework 3.1 Framework of i.MX8MP, i.MX8MQ and i.MX8MM   Decoding Process between the framework, vendor decoder component and kernel as follows on i.MX8MP, i.MX8MQ and i.MX8MM. 4.png   3.2 Framework of i.MX8QM   Decoding Process between the framework, vendor decoder component and kernel as follows on i.MX8QM. 5.png stream on: start to decode/encode stream off: stop decoding/encoding qbuf:  Queue the v4l2buffer filled in the decoder into the buffer queue created in the vpu driver so that the VPU can obtain it for decoding. dqbuf: When the VPU decoding is completed, the buffer will be dequeueed so that the decoder/codec can continue to be used. 4. Decode Video and Display Process 4.1 i.MX8MM and i.MX8MP   For i.MX8MP and i.MX8MM, G2D is used to composite layers, and the buffer transmission is completed between Decoder and SurfaceFlinger through the BufferQueue mechanism. The process as follows: 6.png  BufferQueue: As the producer of BufferQueue, decoder provides the decoded buffer, and SurfaceFlinger, as the consumer of BufferQueue, composite the decoded video layer.  Display: Use G2D to composite Android layer and video layer on i.MX8MP. SurfaceFlinger will hand over all layers to Display HAL, and then composite them into the framebuffer by G2D. 4.2 i.MX8MQ   For i.MX8MQ, GPU3D is used to composite Android UI, DCSS is used to composite overlay and android UI. And the buffer transmission is completed between Decoder and SurfaceFlinger through the BufferQueue mechanism. The processing is as follows: 7.png BufferQueue: Decoder provides the decoded buffer as producer. SurfaceFlinger as the cosumer of BufferQueue.  Display: Using GPU3D to composite Android UI layers. And video layer will be submitted to DCSS (Display Controller) through Display HAL. DCSS:  Responsible for combining video overlay and Android UI for display. GLESRenderEngine: SurfaceFlinger calls the opengl interface through GLESRenderEngine to complete rendering composite. 4.3 i.MX8QM For i.MX8QM, GPU3D or DPU are used to composite. And the buffer transmission is completed between Decoder Filter Component and SurfaceFlinger through the BufferQueue mechanism. The processing as follows: 8.png Decoder Filter:  Because GPU3D cannot directly composite TILED type layers, it needs to be converted into Linear through Decoder Filter first, and then handed over to SurfaceFlinger. BufferQueue:  Decoder Fillter will receive the outputbuffer from Decoder. And Filter will be as producer of BufferQueue to provide the buffer. SurfaceFlinger will be as consumer of BufferQueue to consume the buffers. GLESRenderEngine: SurfaceFlinger calls the opengl interface through GLESRenderEngine to complete rendering composite. Note:  On i.MX8QM,  Not all situations are composited using the GPU3D. Please refer to the table below.   9.png 4.4 DRM Widevine (Secure Decoder)   Now we have enabled DRM Widevine on i.MX8MP/i.MX8MQ/i.MX8QM so that secure video can be played.  For i.MX8MP, Use OEMCrypto trusted application to decrypt the encrypted stream. And Use RDC/CSU to protected hardware for secure pipeline. The framework is as follows: 10.png   For i.MX8MQ, Use OEMCrypto trusted application to decrypt the encrypted stream. And Use RDC/CSU to protected hardware for secure pipeline. The framework is as follows: 11.png   12.png   OEMCrypto: It is a library as tipc client to send the encrypted data to Trusty OS. OEMCrypto Trusted Application: Used to decrypt the protected data into secure memory and send it to Media Framewrok. Secure Framebuffer: It is allocated from secure heap through libdmabufheap. Secure heap: Used to allocate secure memory.  Resources in domain2 can read and write secure memory, but cannot write normal memory. Resources in domain0 can write to secure memory, but cannot read normal memory. When playing secure video, VPU, GPU2D, and lcdif will be in domain2. Or they are in domain0. Resource Domain Controller (RDC): It provides support for the isolation of destination memory mapped locations such as peripherals and memory to a single core, a bus master, or set of cores and bus masters.  CSU Central Security Unit (CSU) : 1. Peripheral Access Policy - the appropriate bus master privilege and identity are required to access each peripheral. 2. Masters Privilege Policy - the CSU overrides the bus master privilege signals (secure/non-secure).  Configure the VPU so that it can only be accessed by the secure world. For i.MX8QM, also use OEMCrypto trusted application to decrypt the encrypted stream. But related hardware and memory are protected through the secure partition created by SCU. The framework is as follows: 13.png OEMCrypto: It is a library as tipc client to send the encrypted data to Trusty OS. OEMCrypto Trusted Application: Used to decrypt the protected data into secure memory and send it to Media Framewrok. Secure Framebuffer: It is allocated from secure heap through libdmabufheap. Secure heap: Used to allocate secure memory.  Only Secure partition can access secure memory. Frimware Loader: It is a trusted application in Trusty OS, It is responsible for loading the encrypted firmware into the specified memory. Secure Partition: Secure partitions are created using SCU and all hardware that needs to be protected are moved to secure partitions to isolate it from non-secure partitions. 5. Encoding Process 5.1 Encoding Process on i.MX8MP 14.png BufferQueue:  MediaFramework will create Surface and create BufferQueue, SurfaceFlinger will serve as the producer to provide the composite layers, and the Encoder component will encode it as the consumer of the BufferQueue. MPEG4Writer:  Responsible for writing the VPU-encoded data to the output file. 5.2 Encoding Process on i.MX8QM and i.MX8MM 15.png   BufferQueue:  MediaFramework will create Surface and create BufferQueue, SurfaceFlinger will serve as the producer to provide the composite layer, and the Encoder Filter component will be as the consumer of the BufferQueue. MPEG4Writer:  Responsible for writing the VPU-encoded data to the output file. 6. Buffer transfer and management 6.1 Transfer and release process of Input Buffer 16.png Input Buffer allocation: It is allocated in CCodecBufferChannel, and it is used to allocate inputbuffer and recycled. InputManager: Before the queue buffer into the decoder, it will be registered with the InputManager. When the VPU is finished using it, the InputManager will be notified to release it. Use of InputBuffer:  After the buffer is queued into the decoder, the input buffer information will be copied to v4l2buffer so that the VPU can use it. Note: For DRM Secure decoder, the VPU will only use the paddr of inputbuffer. 6.2 Transfer and release process of Output Buffer   17.png   Output Buffer allocation:  When use Surface for output, It is allocated through BufferQueueAllocator that is created in CCodecBufferChannel. It is used to allocate outputBuffer.  Management: When use Surface, OutputBuffer is managed through the BufferQueue mechanism. When the VPU filled the outputBuffer with data that can be displayed, it will notify the Consumer to acquire the buffer. Use of OutputBuffer: In the decoder, the output buffer information will be copied to v4l2buffer so that the VPU can use it. Note:  For situations where OutputSurface is not used, GrallocAllocator is used by default instead of BufferQueueAllocator. 6.3 Buffer Management of Encoder   For screen recording situations, the Encoder's buffer transfer is also managed through the BufferQueue mechanism. After SurfaceFlinger is producer to fill the GraphicBuffer of BufferQueue, the encoder is as consumer to encode the composited data.  

Enabling Dual Display in Ubuntu with the i.MX53 Quick Start Board Here you will learn how to enable two displays in a Ubuntu system running in an iMX53 Quick Start Board. We assume here that you already have a micro SD card with a valid Ubuntu image (including uboot, Linux kernel and Ubuntu filesystem). You can use the original SD card that comes with the i.MX53 Quick Start Board, which brings an image of Ubuntu or, if you do not have the original SD card, you can reproduce it by downloading Ubuntu binaries package (L2.6.35_MX53_ER_1101_IMAGE) from Freescale iMX53qsb download area. You will also need to update U-boot and kernel binaries in the SD card with more recent images. You can find the most recent binaries (L2.6.35_MX53_ER_1109_IMAGE_) from Freescale iMX53qsb download area as well. Introduction To enable dual display, you need to perform two tasks: Enable two displays at kernel level Configure your Xorg server accordingly Enabling Two Displays at Kernel Level To enable two displays at kernel level means to map one display interface to fb0 device and the other to fb1 device. So first thing is to choose which interface will be the primary one, mapped as fb0. As an example, we consider the VGA interface as primary in this tutorial. Second thing is to choose one of the other available external video interfaces to be secondary, mapped as fb1 device in the system. We consider the 4.3" seiko LCD display in this tutorial as the secondary interface. Once chosen primary and secondary interfaces, we need to configure kernel video arguments accordingly. The arguments are available in specific variables in the U-boot that comes with the Ubuntu binaries. You can see them (HDMI, VGA, etc.) by printing the U-boot environment variables from the U-boot shell, but you will probably not find those variables in other versions of U-boot and their contents will probably need to be adapted to the kernel version in use, as arguments recognized by kernel modules varies considerably between kernel versions. You can always refer to the Linux Release Notes documents for video arguments. It's available for each Linux BSP that can be found on the Freescale website. For the 1109 BSP, we have the following video arguments (extracted from i.MX53_START_Linux_BSP_Release_Note.pdf that comes with L2.6.35_11.09.01_ER_docs.tar.gz downloaded from here - IMX53_1109_LINUXDOCS_BUNDLE😞 VGA: video=mxcdi1fb:GBR24,VGA-XGA di1_primary vga SEIKO LCD: video=mxcdi0fb:RGB24,SEIKO-WVGA di0_primary Both are considered primary, because these are the arguments for single display setups. Now that we have video arguments for both desired interfaces, we only need to merge them together removing the primary argument from the one that is the secondary. In our case, we need to pass the following arguments to the kernel: video=mxcdi1fb:GBR24,VGA-XGA di1_primary vga video=mxcdi0fb:RGB24,SEIKO-WVGA For this, we can add these arguments to one of the variables that are used in the boot process. We can add the content to bootarg_base, for instance. In the U-boot command line, execute following commands to setup the environment: setenv vga_and_seiko 'video=mxcdi1fb:GBR24,VGA-XGA di1_primary vga video=mxcdi0fb:RGB24,SEIKO-WVGA' setenv bootargs_base 'setenv bootargs console=ttymxc0,115200 vga_and_seiko' saveenv After applying a reset and booting the board, you shall have both interfaces enabled, but the secondary will not be used by the Xorg server until we complete the next step. Configuring Xorg Server Now that we have two video interfaces properly configured and mapped to /dev/fb0 and /dev/fb1 devices, we need to tell Xorg server how to use them. Here is an example of xorg.conf file that you can use to replace the default one, found at /etc/X11: Section "InputDevice" Identifier     "Generic Keyboard" Driver          "kbd" Option          "XkbRules"     "xorg" Option          "XkbModel"     "pc105" Option          "XkbLayout"     "us" EndSection  Section "InputDevice" Identifier     "Configured Mouse" Driver          "mouse" Option          "CorePointer" EndSection  Section "Device" Identifier     "i.MX Accelerated Framebuffer Device 0" Driver          "imx" Option          "fbdev"               "/dev/fb0"  # This option only recognized when "mxc_epdc_fb" frame buffer driver in # use.  Values are "RGB565" (default, 16-bit RGB), "Y8" (8-bit gray), # and "Y8INV" (8-bit gray inverted). Option          "FormatEPDC"               "Y8INV"  EndSection  Section "Device" Identifier     "i.MX Accelerated Framebuffer Device 1" Driver          "imx" Option          "fbdev"               "/dev/fb1"  EndSection  Section "Monitor" Identifier     "Configured Monitor 0" EndSection  Section "Monitor" Identifier     "Configured Monitor 1" EndSection  Section "Screen" Identifier     "Screen 0" Monitor          "Configured Monitor 0" Device          "i.MX Accelerated Framebuffer Device 0"  # These "Display" SubSection's are needed for working with the # "mxc_epdc_fb" frame buffer driver. SubSection     "Display" Depth     8 Visual     "StaticGray" EndSubSection SubSection     "Display" Depth     16 Visual     "TrueColor" EndSubSection EndSection  Section "Screen" Identifier     "Screen 1" Monitor          "Configured Monitor 1" Device          "i.MX Accelerated Framebuffer Device 1" EndSection  Section "ServerLayout" Identifier     "Xinerama Layout" Screen          "Screen 0" Screen          "Screen 1" RightOf "Screen 0" EndSection  Section "ServerFlags" Option          "Xinerama"          "true" EndSection Results The following picture shows the i.MX 53 QSB running the extended desktop previously configured. You can see the VGA monitor with a Firefox instance and the SEIKO LCD display with a calc instance.

Information about the transition from the NXP Demo Experience to GoPoint for i.MX Application Processors.

GUI Guider version: 1.6.x, 1.7.x, 1.8x LVGL version: v8.x.x Host software requirements: Ubuntu 20.04, Ubuntu 22.04 or Debian 12 Hardware requirements: Evaluation Kit for the i.MX 93 Applications Processor. (i.MX 93 Evaluation Kit | NXP Semiconductors) On this guide we will use the IMX-MIPI-HDMI accessory board to connect the iMX93 with a HDMI Monitor. (IMX-MIPI-HDMI Product Information|NXP) This board is usually provided with the iMX8M Mini and the iMX8M Nano.  Steps: 1. Copy your project from the folder GUI-Guider-Projects to your Linux PC.  2. Build an image for iMX93 using The Yocto Project.    a. Based on iMX Yocto Porject Users Guide set directories and download the repo $ mkdir imx-bsp-6.1.1-1.0.0 $ cd imx-bsp-6.1.1-1.0.0 $ repo init -u https://github.com/nxp-imx/imx-manifest -b imx-linux-langdale -m imx-6.1.1-1.0.0.xml $ repo sync Use distro fsl-imx-xwayland and select machine imx93evk and use this commnad with a build folder name: $ MACHINE=imx93evk DISTRO=fsl-imx-xwayland source ./imx-setup-release.sh - b bld-imx93evk b. Use bitbake command to start the build process. Also, add the -c populate_sdk to get the toolchain. $ bitbake imx-image-multimedia -c populate_sdk  c. Install the Yocto toolchain located on <build-folder>/tmp/deploy/sdk/.  $ sudo sh ./fsl-imx-xwayland-glibc-x86_64-imx-image-multimedia-armv8a-imx93evk-toolchain-6.1-langdale.sh d. Install ninja utility on the build host $ sudo apt install ninja-build e. For Ubuntu 20.04 and Ubuntu 22.04, copy the lv_conf.h file from lvgl-simulator to lvgl $ cp lvgl-simulator/lv_conf.h lvgl/ f. Change the interpreter on build.sh from #!/bin/sh to #!/bin/bash. This is an important step! g. Then, enter to linux folder and use the following commands to make build.sh executable $ dos2unix build.sh $ chmod +x build.sh h. Execute the build.sh $ ./build.sh i. Copy the binary to the iMX93 using a USB or SCP.  2. On the target iMX93 follow these steps. a. On Uboot, use fatls interface device:partition fatls mmc 0:1 (Device 0 : Partition 1) With this command, we will be able to list device tree files. => fatls mmc 0:1 b. Select imx93-11x11-evk-rm67199.dtb and use the command editenv fdtfile  => editenv fdtfile Output example edit: imx93-11x11-evk-rm67199.dtb c. In edit command line put the selected device tree .dtb d. Use saveenv command to save environment and continue with the boot process. e. Finally, run the GUI Application $ ./gui_guider&   I hope this article will be helpful. Best regards, Brian.

      The i.MX6UL/LL/LZ processor supports 2 USB OTG interfaces, USB OTG1 and USB OTG2, and each USB interface can be configured as a device, host or dual role mode. On the EVK board of i.MX6UL/LL, USB OTG1 is designed as dual role mode, and USB OTG2 is designed as HOST mode. This is sufficient for most customers.       However, in actual applications, we may need 2 USB HOSTs, and at the same time, we don’t want to use MicroUSB to USB TYPE-AF cable for Host-Device mode conversion. Therefore, the design of the USB circuit needs to meet such requirements: 1. USB device mode We need a USB device to download the linux image to the flash or SD card on the board. 2. 2 USB HOSTs When the system is working normally, we need the board to support 2 USB HOST. i.MX6UL/LL/LZ has only 2 USB ports. How to design to meet this requirement without increasing the USB HUB? The following scheme is used as a reference, and I hope it will be helpful to customers with similar requirement: 6.png        The logic and application description of this Diagram:: Default—device mode In the process of debugging the software, we need to use the USB OTG interface to download the linux image, so it must work in device mode. What we need to do is: (1). Pull USB OTG ID up to 3.3V (2). The USB OTG D+/D- signal is switched to the MicroUSB connector. (3). The USB OTG VBUS is provided with 5V power from the external PC USB HOST. Usage:        -Use a jumper for Pin 1 and Pin2, USB OTG ID pin will be pulled up to High.        With the operation, SEL pin of USB Muxer is High, and USB signals are switched to port B, and USB differential signals are connected to MicroUSB connector. At the same time, MIC2026-1YM output is disabled. The USB OTG1 VBUS pin of CPU is supplied by VBUS of MicroUSB connector, that is to say, supplied by PC USB HOST.        In this mode, software engineer can use it to download images to flash on board. Normal Work—Host mode After the software debugging is completed, two HOSTs are needed on the board. At this time, we need to switch the USB OTG1 from device to HOST mode. What we need to do is: (1). Pull USB OTG1 ID down to LOW (2). The USB OTG D+/D- signal is switched to the USB Type-AF connector. (3). Board should supply 5V power for USB device connected USB Type-AF connector. Usage:        -Use a jumper for Pin 2 and Pin3, USB OTG ID pin will be pulled down to Low.        With the operation, USB OTG1 ID pin is pulled down to Low, SEL pin of USB Muxer is also LOW, USB signals are switched to Port A, and connected to USB type-AF connector. At the same time, MIC2026-1YM is enabled , OUTA will output 5V , which will supply USB device connected on USB type-AF connector.   [Note] Users need to pay attention to. When using the jumper with PIN1/2/3, the board needs to be powered off. In other words, when switching between device and host, you need to switch off the power, then power on, and restart the board. The solution can also be used for i.MX processors with USB 2.0 interface.   NXP CAS team Wedong Sun 01/15/2021

Introduction This document describes the Spread Spectrum support for displays on i.MX 8QuadMax and i.MX 8QuadXPlus, specific for LVDS display. It describes the underlying HW function, how to enable it and the intended capability. The display controller (DC) subsystem on i.MX 8QuadMax and i.MX 8QuadXPlus uses an AVPLL to generate the reference clock for operation of the LVDS PHYs.  Enabling Spread Spectrum on the reference clock will result in the PHY interfaces being spread as well. This Spread Spectrum feature is controlled by the SCU firmware and can be enabled or disabled by configuring the board file of the SCU firmware porting kit. (The Spread Spectrum feature is added starting from SCFW porting kit V1.2.2 release which can be download from NXP web site “i.MX Software and Development Tool”.) The User Guide will include following content: 1. Introduction ............................................................................ 1 2. Configuration of the frequency modulation ......................... 2 3. Support in SCFW Porting Kit ............................................... 4 4. Modulation Characteristics ................................................... 4 5. Enablement Example ............................................................. 5 6. Revision History .................................................................... 7 For more information, please check the attachment "User Guide of Spread Spectrum for i.MX8QM_QXP Display.pdf".   Rev2.0 Update For SCFW Porting Kit V1.2.5 and later version, please check document "User Guide of Spread Spectrum for i.MX8QM_QXP Display 2.0.pdf" with updated algorithm. Rev2.1 Update For SCFW Porting Kit V1.2.10 and later version, please check document "User Guide of Spread Spectrum for i.MX8QM_QXP Display 2.1.pdf" with fspread value selection feature. Users can choose the percentage of frequency spread from following values: 0%, 0.4%, 1.0%, 1.4%, 2.0%.

In i.MX51 platfrom the PMIC 13892 also has a internal RTC. We can use this RTC instead of the i.mx51 SRTC. Attached was the implementation of it.