This doc will show: i.MX8DXL EVK board without connect  hardware LCD display, using FreeRDP to share screen to remote PC which is in same network,  PC take this shared screen could  run any command on i.MX8DXL EVK board.   HW: i.MX8DXL EVK board,  PC  SW: i.MX8DXL Linux 5.15.32 BSP release, and code change in this doc   1> i.MX8DXL Linux kernel side, just use imx8dxl-evk-lcdif.dts, but did not connect any LCD display,  so Linux kernel could create related drm device, and weston could be start. But on 8DXL EVK board, ENET1_RGMII_TXD3 and ADMA_LCDIF_D03 pin conflict,  so need code change as 5.15.32-imx8dxl-evk-lcdif.dts.diff.   2> yocto/bld/conf/local.conf, add below line, as freerdp depend on ffmpeg. LICENSE_FLAGS_ACCEPTED+="commercial"   3> For i.MX8DXL,  weston use software pixman render, to use NEON optimization,  pixman need switch to  latest 0.42.0,  enter folder (yocto/bld/tmp/work/armv8a-poky-linux/pixman/1_0.40.0-r0/pixman-0.40.0),  git clone (https://github.com/freedesktop/pixman.git) and checkout to latest 0.42.0:  Also chaneg related build flag at bb file as pixman_0.40.0.bb.diff.     4> Default  freerdp need switch to 2.8.0, enter folder (yocto/bld/tmp/work/armv8a-poky-linux/freerdp/1_2.4.1-r0/git),  git clone (https://github.com/FreeRDP/FreeRDP.git), check out to 2.8.0; And to use neon accelerate freerdp related function, such as color space conversion, image codec encoding ,  need apply patch  freerdp-codechange-neon.diff. And related bb file compile flag change as freerdp_2.4.1.bb.diff     5> bitbake -c compile ffmpeg bitbake -c install ffmpeg   bitbake -c compile pixman  bitbake -c install pixman    bitbake -c compile  freerdp bitbake -c install  freerdp   Copy  generated new libs to default released i.MX8DXL rootfs, make sure ffmeg , pixman, freerdp related libs are  from your build, for example: libfreerdp-client2.so.2 -> libfreerdp-client2.so.2.8.0 libfreerdp2.so.2 -> libfreerdp2.so.2.8.0 libwinpr-tools2.so.2 -> libwinpr-tools2.so.2.8.0 libwinpr2.so.2 -> libwinpr2.so.2.8.0   6> i.MX8DXL Linux rootfs:  file /etc/xdg/weston/weston.ini,  change start-on-startup to true: [screen-share] command=/usr/bin/weston --backend=rdp-backend.so --shell=fullscreen-shell.so --no-clients-resize --rdp-tls-cert=/etc/freerdp/keys/serve start-on-startup=true   7> i.MX8DXL Linux OS, run below cmd: winpr-makecert -path $PWD copy generated files to /etc/freerdp/keys/server.crt and /etc/freerdp/keys/server.key   8> reboot i.MX8DXL EVK board,  make sure  EVK board and PC in the same network; check i.MX8DXL Linux OS , there are two process name as "weston", one process is weston rdp backend to share screen to PC.   9> PC side, get wfreerdp.exe from  https://github.com/FreeRDP/FreeRDP/wiki/Prebuilds PC side run cmd as: wfreerdp.exe /v:ipaddress_of_8DXLEVK     Reference: 1>https://www.nxp.com/design/software/embedded-software/i-mx-software/embedded-linux-for-i-mx-applications-processors:IMXLINUX#design-resources 2>https://github.com/FreeRDP 3>https://github.com/freedesktop/pixman 4>https://github.com/DLTcollab/sse2neon  

On behalf of Gopise Yuan. This is an update for the DRM framebuffer capture tool I used to share with the team. Some enhancement added. Might be useful for debugging some display related issues.   Some special notes: Behavior of DRM subsystem is different between L4.x and L5.x. For L4.x, you can capture the RGB buffer without any problem. But, there’s no API for YUV (multi-plane) buffer. To capture YUV, need to apply “kernel_0001-drm-Add-getfb2-ioctl_L4.14.98.patch”. For L5.x, mapping/capturing the internal buffer is not allowed by default due to security reason. A simple change in “0001-drm-enable-mapping-of-internal-object-for-debugging_L5.x.patch” can disable this guard. Capture raw data only. RGB and YUV (packed/planar, 420/422) supported. Support de-tile on “Amphion tile” (VPU, NV12 only) and “Super tile” (Vivante GPU). Please use “-t” to enable this. Other tile might not be supported. This is a static linked binary. You can run it on any ARM64 based Linux/Android system in theory (prerequisites in item #1). If you need source code, come to me.   To get more details on how to use it, use “-?” option: DRM screen capture DRM based screen capture program Usage:     ./drmfbcap [OP] [ARG] [OP] OPeration (optional):     -v Show version.     -? Show help information.     -i Show information about target DRM device only (no capture).     -t Perform de-tile for tile format.     -d DRM device to open. [ARG] should contain the path to the device node. Default: '/dev/dri/card0'     -o Output folder. [ARG] should contain the path to the output folder. Default: '.'     -p Specific plane # to capture. [ARG] should contain the plane number. If no '-p' specified, capture all planes   Example:     ./drmfbcap   Capture all planes on default DRM device.     ./drmfbcap -d /dev/dri/controlD64   Capture all planes on '/dev/dri/controlD64' device.     ./drmfbcap -p 44 -t -o /sdcard   Capture plane 44, do de-tile after capture and then output to /sdcard/.   Raw buffer capture will be done for each enabled/target plane and one file for each. Captured file will be saved to './' if not specified. --- By Gopise, 2022/08   Updated_2023_10_16: continuous capture (repeat mode) support with this link: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/DRM-screen-capture-tool/ta-p/1725363

On behalf of Gopise Yuan. In case some customer may make mistake in mechanical design, they may need to flip/mirror the screen. DPR in DPU can do this in a simple way. This patch demonstrate how to enable VFLIP and HFLIP in DPR to do a V+H flip (=180 rotate) of the screen

This is simple known-how for how to implement "boot animation" with DRM under i.MX8/X + Linux:   Code to refer to: ========================================================================= 1. kmscube: Either open source one or the customized on for i.MX will be OK: https://cgit.freedesktop.org/mesa/kmscube/ https://source.codeaurora.org/external/imx/kmscube-imx/ 2. Android display HAL: KmsDisplay.cpp   Known-how: ========================================================================= 1. Only one application can grab the master role of the DRM device. If need to control DRM from two applicaiton simultanously, possible solution:     A, Use "controlD" node instead of "card" node in /dev/dri/. This requires L4.14 or before. This device node was removed by two commits in L4.14.x:           8a357d10043c75e980e7fcdb60d2b913491564af           6449b088dd51dd5aa6b38455888bbf538d21f2fc     Can be brought back by reverting these two commits in L4.14.98.     B, Use framebuffer emulator to emulate a FB device (/dev/fb0). (not recommended due to lack of vsync). 2. Some kernel functions will re-config the DRM device during boot. This will cause display abnormal after user application has configured the DRM device. Better to disable these kernel features:       CONFIG_DRM_FBDEV_EMULATION       CONFIG_FRAMEBUFFER_CONSOLE 3. Use atomic mode of KMS API instead of legacy mode for any dynamically screen drawing application, such as video, game and etc. Atomic mode will have much better performance compare to legacy mode. The kmscube has sample code for both mode. 4. Better to do commit checking before doing any real commit, especially when doing display during boot. Sometimes some internal component in DRM is not fully ready after card device is present.       DRM_MODE_ATOMIC_ALLOW_MODESET 5. If video playback will be used, some points to remind:     a, Sample code for direct video decoding (in unit-test): imx-test/test/mxc_v4l2_vpu_test/     b, VPU in i.MX8/X only support tiled NV12 output and it has pixel alignment requirement (128). Need to use CPU or G2D to do un-tile, CSC and cropping. Sample code: <android>/vendor/nxp/fsl_imx_omx/OpenMAXIL/src/component/v4l2_common/G2dProcess.cpp If using G2D under Linux, it will support un-tile directly (through OpenCL internally).

This is based on L5.10.35 BSP where you have to install QT static build: Qt 5.15 static build: Assuming your sysroot is at "/sysroot-cross" and your toolchain is at "/Toolchain" your qt-source is at /Qt-5.15 PATH=/sysroot-cross/bin:/sysroot-cross/sbin:/Toolchain/bin mkdir /Qt-5.15/mkspecs/qws/linux-imx6-g++ create in this dir the textfile "qmake.conf" with this content: ####################### snip qmake.conf ############################## include(../../common/linux.conf) include(../../common/qws.conf) # modifications to g++.conf QMAKE_CC                = arm-linux-gnueabi-gcc QMAKE_CFLAGS            = -pipe -isystem /sysroot-cross/include -isystem /sysroot-cross/usr/include QMAKE_CXX               = arm-linux-gnueabi-g++ QMAKE_CXXFLAGS          = -pipe -isystem /sysroot-cross/include -isystem /sysroot-cross/usr/include QMAKE_INCDIR            = /sysroot-cross/include /sysroot-cross/usr/include QMAKE_LIBDIR            = /sysroot-cross/lib /sysroot-cross/usr/lib QMAKE_LINK              = arm-linux-gnueabi-g++ QMAKE_LINK_SHLIB        = arm-linux-gnueabi-g++ QMAKE_LFLAGS            = -L/sysroot-cross/lib -L/sysroot-cross/usr/lib -Wl,-rpath-link -Wl,/sysroot-cross/lib QMAKE_LFLAGS           += -Wl,-rpath-link -Wl,/sysroot-cross/usr/lib #Opengl QMAKE_INCDIR_OPENGL = /Vivante/include QMAKE_INCDIR_OPENGL += /Vivante/include/GL QMAKE_INCDIR_OPENGL += /Vivante/include/EGL QMAKE_INCDIR_OPENGL += /Vivante/include/GLES2 QMAKE_LIBDIR_OPENGL = /Vivante/lib QMAKE_INCDIR_OPENGL_ES1 = $$QMAKE_INCDIR_OPENGL QMAKE_LIBDIR_OPENGL_ES1 = $$QMAKE_LIBDIR_OPENGL QMAKE_INCDIR_OPENGL_ES1CL = $$QMAKE_INCDIR_OPENGL QMAKE_LIBDIR_OPENGL_ES1CL = $$QMAKE_LIBDIR_OPENGL QMAKE_INCDIR_OPENGL_ES2 = /Vivante/include QMAKE_INCDIR_OPENGL_ES2 += /Vivante/include/EGL QMAKE_INCDIR_OPENGL_ES2 += /Vivante/include/GLES2 QMAKE_LIBDIR_OPENGL_ES2 = $$QMAKE_LIBDIR_OPENGL QMAKE_INCDIR_EGL = $$QMAKE_INCDIR_OPENGL_ES2 QMAKE_LIBDIR_EGL = $$QMAKE_LIBDIR_OPENGL QMAKE_LIBS_EGL = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL_ES2 = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL_QT = -lEGL -lGAL -lGLESv2 -lGLES_CM QMAKE_LIBS_OPENGL_ES1 = QMAKE_LIBS_OPENGL_ES1CL = # modifications to linux.conf QMAKE_AR                = arm-linux-gnueabi-ar cqs QMAKE_OBJCOPY           = arm-linux-gnueabi-objcopy QMAKE_STRIP             = arm-linux-gnueabi-strip QMAKE_CFLAGS_RELEASE   = -pipe -isystem /sysroot-cross/include -isystem /sysroot-cross/usr/include load(qt_config) ####################### snip qmake.conf ############################## create in the same dir the text file "qplatformdefs.h" ####################### snip qplatformdefs.h ############################## #include "../../linux-g++/qplatformdefs.h" ####################### snip qplatformdefs.h ############################## now goto dir /Qt-5.15 cd /Qt-5.15 call configure with ./configure -opensource -confirm-license -release -no-rpath -no-fast \     -no-sql-ibase -no-sql-mysql -no-sql-odbc -no-sql-psql -no-sql-sqlite2 \     -no-qt3support -no-mmx -no-3dnow -no-sse -no-sse2 -no-sse3 -no-ssse3 \     -no-sse4.1 -no-sse4.2 -no-avx -no-optimized-qmake -no-nis -no-cups -pch \     -reduce-relocations -force-pkg-config -prefix /usr -no-armfpa -make libs \     -nomake docs -little-endian -embedded armv6 -qt-decoration-styled \     -depths all -xplatform qws/linux-imx6-g++ -iconv -largefile -qt-gfx-linuxfb \     -qt-gfx-multiscreen -qt-mouse-pc -qt-mouse-linuxinput -qt-libpng \     -plugin-gfx-directfb -system-zlib -no-accessibility -no-gfx-transformed \     -no-gfx-qvfb -no-gfx-vnc -no-kbd-tty -no-kbd-linuxinput -no-kbd-qvfb \     -no-mouse-linuxtp -no-mouse-tslib -no-mouse-qvfb -no-libmng -no-libtiff \     -no-gif -no-libjpeg -no-freetype -no-stl -no-glib -no-openssl -no-egl \     -no-xmlpatterns -no-exceptions -no-multimedia -no-audio-backend -no-phonon \     -no-phonon-backend -no-webkit -no-script -no-scripttools -no-svg -no-script \     -no-declarative -no-sql-sqlite -no-qdbus -no-opengl -static -nomake tools \     -nomake examples -nomake demos when configuring is finished call make after a looong time, when everything goes right, we have a staticly compiled Qt. DO NOT call "make install". We will install manually: copy from /Qt-5.15/bin the files moc, uic, rcc and qmake to somewhere in PATH, eg. /sysroot-cross/bin copy the contents of dir /Qt-5.15/mkspecs to /sysroot-cross/usr/mkspec copy the contents of dir /Qt-5.15/plugins to /sysroot-cross/usr/plugins copy the contents of dir /Qt-5.15/include to /sysroot-cross/usr/include copy the contents of dir /Qt-5.15/lib to /sysroot-cross/usr/lib Test application camtest: if you don't have/want directfb plugin remove from camtest.pro the lines LIBS += -L/sysroot-cross/usr/plugins/gfxdrivers QTPLUGIN += QDirectFBScreen and the lines from main.cpp #include <QtPlugin> Q_IMPORT_PLUGIN(qdirectfbscreen) generate makefile by typing /sysroot-cross/bin/qmake -spec /sysroot-cross/usr/mkspecs/qws/linux-imx6-g++ camtest.pro then make you should set and activate your framebuffers with this script ################# snip ################################ fbset -fb /dev/fb0 -g 1024 768 1024 2304 16 echo -n 0 > /sys/class/graphics/fb0/blank fbset -fb /dev/fb1 -g 1024 768 1024 1536 32 echo -n 0 > /sys/class/graphics/fb1/blank modprobe galcore modprobe uvcvideo modprobe mxc_v4l2_capture ################# snip ################################ if you use directfb then your /etc/directfbrc file should look like this: ######################## snip /etc/directfbrc ############# system=fbdev fbdev=/dev/fb1 mode=1024x768 depth=32 pixelformat=ARGB no-cursor window-surface-policy=systemonly ######################## snip /etc/directfbrc ############# to start the application with directfb: ./camtest -qws -display directfb without directfb using linuxfb: ./camtest -qws -display linuxfb:/dev/fb1 Notes about application: 1. The application shows 2 webcams in background-framebuffer (BG-FB). The foreground-framebuffer (FG-FB) shows the qt-gui. FG-FB is configured to be fully opaque and uses color-keying. On the BG-FB one cam is overlayed on the other cam using IPU. Optimization possibilities: the app copies the frames from the cams with memcpy. This wouldn't be necessary, when the kernel usb-webcam interface (uvc) would support V4L2_MEMORY_USERPTR method. through this way, you could pass the mapped IPU mmapped inbufs directly to v4l2 output buffers. If you get errors like NOSPC (-28) from uvc, this is a limitation of USB. My board is a MX6QSabre, where the two webcams are connected to the same usb-controller. With both webcams I had to limit the frame size to 320x250 and 160x120 at 25Hz. You might try higher res if you have other type of webcams (not usb). Have fun  

Use case: iMX8QXP system can be a video input source to another system.   Hardware Pins： LCDIF_D00 ~ LCDIF_D07 LCDIF_CLK LCDIF_VSYNC LCDIF_HSYNC LCDIF_EN   Reference patch： It is based on L5.4.70_2.3.0 GA BSP.  File: L5.4.70_2.3.0-iMX8QXP-LCDIF-add-YUV422-8-bits-output.patch Customer can change the timing parameters in file "panel-lcdif-yuv422.c" as needed, the default timing is a 1280x720 P30 mode: static const struct display_timing yuv422_lcd_timing = {     .pixelclock = { 74250000, 74250000, 74250000 },     .hactive = { 1280, 1280, 1280 },     .hfront_porch = { 220, 220, 220 },     .hback_porch = { 110, 110, 110 },     .hsync_len = { 40, 40, 40 },     .vactive = { 720, 720, 720 },     .vfront_porch = { 20, 20, 20 },     .vback_porch = { 5, 5, 5 },     .vsync_len = { 5, 5, 5 },     .flags = DISPLAY_FLAGS_DE_HIGH, };   Test application: drm_test_yuv.zip: it can set framebuffer to UYVY mode, in this case, no CSC is needed, the data in framebuffer memory will be same as output on display data interface. drm_test_rgb.zip: it can set framebuffer to RGBA mode, in this case, RGB to YUV CSC is needed, application can draw RGB data into framebuffer as normal, the LCDIF will convert it to YUV422 format on the fly, then output the YUV data to display interface.    

This doc share one OpenGL ES sample code, it is running on i.MX8 MEK board with QNX SDP7.1. HW: i.MX8 MEK board, HDMI display SW: QNX SDP7.1, i.MX8 MEK board BSP, and this sample code   This sample code will draw 3D object model, and with some animation. Reference: https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-processors/i-mx-8-family-arm-cortex-a53-cortex-a72-virtualization-vision-3d-graphics-4k-video:i.MX8 https://github.com/NXPmicro/gtec-demo-framework https://github.com/syoyo/tinyobjloader-c https://github.com/nothings/stb https://3dhaupt.com/futuristic-car-game-ready-download/ https://wallpapersafari.com/w/Y5JZNh https://www.pngwing.com/en/free-png-ysaus https://www.shadertoy.com/view/Ms2SWW#

BSP: L5.4.47-2.2.0-rc2 Board: imx8QM B0 HW:  LVDS2HDMI , MIPIDSI2HDMI. It is the porting of i.MX8QM dpu loopback to isi. to the 5.4.y, with the addition of the MIPI-DSI loopback and the HDMI loopback.  Overview of the DC capture configuration: For enabling the capture: only DC 0 Stream 0  and DC 1 Stream 1 can be captured The pixel link Master address should be set to 3 because the Receiver Address at ISI is 3 and can't be changed. To continue displaying the stream, the Receiver Address at LVDS and DSI or HDMI should be changed to 3. It is possible to change the RA by using GPIO of the modules.   Patches: Create V4L2 device enabling the capture of by the ISI of DC loop-backs. Enable ISI capture from DSI 0 / LVDS 1 in 1920x1080 (at the same time.) Enable ISI capture from HDMI in 2840x2160 (half with even pixel) in 1920x2160. While capturing with the ISI, the captured screen continue to be displayed. Remark: Ov5640 cameras are also enabled in the same dtb. So 4 stream in 1920x1080 can be captured at the same time. Installation and gstreamer command: See readme

The i.MX8QuadMax SMARC System On Module integrates Dual Cortex A72 + Quad Cortex A53 Cores, Dual GPU systems, 4K H.265 capable VPU dual failover-ready display controller based i.MX8 QuadMax SoC with on SOM Dual 10/100/1000 Mbps Ethernet PHY, USB 3.0 hub and IEEE 802.11a/b/g/n/ac Wi-Fi & Bluetooth 5.0 module.

Header 1 Header 2 Video rendering gst-launch videotestsrc ! mfw_v4lsink Audio rendering gst-launch audiotestsrc ! alsasink WAV Audio rendering gst-launch filesrc location=test.wav ! wavparse ! alsasink Video rendering selecting caps gst-launch videotestsrc ! capsfilter name='video/x-raw-yuv,format=(fourcc)I420' ! mfw_v4lsink gst-launch videotestsrc ! 'video/x-raw-yuv,format=(fourcc)I420' ! mfw_v4lsink

GStreamer has a simple feature to enable tracing, allowing the developer to do basic debugging. These can be done in two ways: Adding the parameter --gst-debug=LIST to the pipeline (a pipeline is a executed gst-launch command) Prepending the environment variable GST_DEBUG=LIST' LIST is a a comma-separated argument, indicating the GStreamer elements to trace. For example, if one needs to trace the sink element      $ GST_DEBUG=*sink*:5 gst-launch playbin2 uri=file:///sample.avi or      $ gst-launch playbin2 uri=file:///sample.avi --gst-debug=*sink*:5 Both commands produces the same log. In case want to trace for than one element, so can simple add the <element>:5, for example      $ GST_DEBUG=mfw_v4lsink:5,vpudec:5 gst-launch playbin2 uri=file:///sample.avi The number 5 indicates the log category, where 5 is the highest (the most verbose log you can get) and 0 produces no output (5=LOG, 4=DEBUG, 3=INFO, 2=WARN, 1=ERROR). Log can be huge in each pipeline run. One way to filter it is using the grep command. Before grepping, one needs to redirect the standard error to the standard output (GStreamer log goes always to stderr), so      $ GST_DEBUG=mfw_v4lsink:5,vpudec:5 gst-launch playbin2 uri=file:///sample.avi 2>&1 | grep <filter string> In case the log needs to be shared, it is important to remove the 'color' of the log, again, one just needs to add the parameter --gst-debug-no-color or prepend the env variable GST_DEBUG_NO_COLOR=1 ----- More shell variables that GStreamer react, can be found here https://developer.gnome.org/gstreamer/0.10/gst-running.html

gst-inspect is a tool to to get documentation about GStreamer elements. Pipeline Check installed GST elements gst-inspect | tail -1 Check installed FSL GST elements gst-inspect | grep imx Element documentation gst-inspect <gst element>

[中文翻译版] 见附件   原文链接： https://community.nxp.com/docs/DOC-345148 

There is no Freescale GStreamer element which does the JPEG decoding, so we must rely on a standard one, like 'jpegdec'. In case your Linux system was built using LTIB, in order to have the jpegdec element included on the gst-plugin-good, follow these steps: On the LTIB menuconfig, make sure the following packages are selected: gstreamer-plugins-good libjpeg libpng Remove the configure parameters '--disbale-libpng' and '--disable-jpeg' on the file './dist/lfs-5.1/gst-plugins-good/gst-plugins-good.spec' Rebuild and flash your board (or SD card) again. Image display VSALPHA=1 gst-launch filesrc location=sample.jpeg ! jpegdec ! imagefreeze ! mfw_isink Important: non 8 pixel aligned width and height is treated as not supported format in isink plugin.

This doc show: on i.MX6Q SabreSD board, configure ov5640 sensor(parallel or MIPI) output 5MP(2592x1944) RAW(Bayer) data at 15fps,and i.MX6Q IPU capture RAW RGB data, and i.MX6Q GPU debayer RAW data then display image. HW: i.MX6Q-SabreSD board, ov5640 sensor. SW: Linux 4.14.98_2.0.0 BSP, and patches in this doc. Configure at camera sensor side A Bayer filter is a color filter array (CFA) for arranging RGB color filters on a square grid of photosensors. The filter pattern is 50% green, 25% red and 25% blue, hence is also called BGGR, RGBG ,GRGB, or RGGB. The ov5640 has an image array capable of operating at up to 15 fps in 5 megapixel (2592x1944) resolution. OV5640 support output formats: RAW(Bayer), RGB565/555/444,CCIR656, YUV422/420, YCbCr422, and compression. To make ov5640 output 5MP RAW data at 15fps, check my patch imx6_ov5640_dvp_mipi_raw_capture_driver-4.14.98_2.0.0.diff which apply on i.MX Linux 4.14.98_2.0.0 BSP kernel code: Parallel interface ov5640, use ov5640_raw_setting[] array of drivers/media/platform/mxc/capture/ov5640.c. This register setting is come from ov5640 software application note and data sheet. MIPI interface ov5640, use ov5640_mipi_raw_setting[] array of drivers/media/platform/mxc/capture/ov5640_mipi.c. This register setting is combine setting of original code (remove ISP register setting), plus PLL register setting for MIPI interface, plus some data format register setting. Configure at i.MX6Q side The i.MX6Q IPU camera port(CSI-2 module) support data format include Raw(Bayer), RGB, YUV 4:4:4, YUV 4:2:2 and grayscale, up to 16 bits per value. Below is camera data routing for i.MX6Q:    Below is i.MX6Q IPU block daigram: The CSI-2 of IPU which is responsible for synchronizing and packing the video (or generic data) and sending it to other blocks. The video data received by CSI-2, could be sent to three other blocks: SMFC, VDI, IC. For RAW (Bayer) data capture, should go through path like this: CSI-2-->SMFC-->IDMAC-->DDR memory It means RAW data is received as generic data, see IPU_PIX_FMT_GENERIC in my patch, and IPU cannot process this kind data, it is just received to DDR memory. For MIPI interface camera, need note is i.MX6 side MIPI D-PHY clock must be calibrated to the actual clock range of the camera sensor’s D-PHY clock and the calibrated value must be equal to or greater than the camera sensor clock, detail see  AN5305. Take MIPI ov5640 as example: Pixel clock = 2592x1944x15fpsx(1/2 cycle/pixel)x1.35 blank interval = 51MHZ MIPI data rate = 51MHZ x 16 bit = 816Mb/s so 816/2/2*2 is 408MHZ is i.MX6 side D-PHY clock. Here due to one bayer pixel is 8bit, and i.MX6 MIPI data bus is 16 bit, so above use 1/2 cycle/pixel. And check ov5640_mipi_raw_setting[], you will got the sensor side D-PHY clock is about 672/2 = 336MHZ. And check AN5305, register MIPI_CSI2_PHY_TST_CTRL1 of i.MX6 need set as 0xC, but here i still keep it as default BSP value 0x14.   3.Capture test code I changed unit test mxc_v4l2_capture.c to capture the RAW data and save it to file. Check my patch imx6_ov5640_raw_captupre_test_4.14.98_2.0.0_ga.diff which apply on i.MX  Linux 4.14.98_2.0.0 BSP unit test code. Note the usage is: ./cap.out -c 1 -i 1 -fr 15 -m 6 -iw 2592 -ih 1944 -ow 2592 -oh 1944 -f BA81 -d /dev/video1 savefile.dmp parameter -i 1 means use CSI to MEM mode /dev/video1 is MIPI ov5640, /dev/video0 is parallel ov5640   4.Display RAW data The RAW data cannot be displayed directly, debayer process is needed to get complete red, green, blue color for each pixel. The debayer process if run on CPU, will cost much CPU time. To save CPU time, debayer could done by GPU. The method is, captured RAW data upload to GPU as texture , then GPU will do the debayer, then full color of each pixel will be got, then display it. To upload RAW camera data to GPU with zero memory copy, i will use i.MX6Q GPU extension GL_VIV_direct_texture. It create a texture with direct access support. API glTexDirectVIVMap,  which support mapping a user space memory or a physical address into the texture surface. The API glTexDirectVIVMap need logic and physical address of data buffer, so i will allocate data buffer from /dev/mxc_ipu, it is dma-buffer also get logic/physical address of buffer, then queue it as USERPTR to ipu v4l2 capture driver, after dequeue got RAW camera data, pass it to GPU for debayer. GPU side, I will use OpenGL shader code from "Efficient, High-Quality Bayer Demosaic Filtering on GPUs". Check my patch imx6-5640-debayer-testcode-gpusdk-5.2.0.diff which apply on i.MX GPU SDK 5.2.0 code. Note, here i only do is debayer, no extra process.   Known issue One thing is ov5640 output 5MP at 15fps, compare with output 5MP at 5fps, there are more noise of camera data at 15fps case. My debug found is , this noise seems come from ov5640 itself.   Reference: a>https://www.nxp.com/webapp/Download?colCode=IMX6DQRM b>https://www.nxp.com/webapp/Download?colCode=L4.14.98_2.0.0_MX6QDLSOLOX&appType=license c>https://github.com/NXPmicro/gtec-demo-framework d>https://www.nxp.com/docs/en/application-note/AN5305.pdf e>ov5640 data sheet f>ov5640 software application note g>Efficient, High-Quality Bayer Demosaic Filtering on GPUs https://www.semanticscholar.org/paper/Efficient%2C-High-Quality-Bayer-Demosaic-Filtering-on-McGuire/088a2f47b7ab99c78d41623bdfaf4acdb02358fb

If someone wants to use the OpenGL ES 2.0 extension "GL_OES_vertex_array_object", the macro "GL_GLEXT_PROTOTYPES" must be defined in his program first. Then he can get the extension program location by calling the API eglGetProcAddress.  Here is an example to use this extension. #define GL_GLEXT_PROTOTYPES PFNGLGENVERTEXARRAYSOESPROC glGenVertexArraysOESv; PFNGLBINDVERTEXARRAYOESPROC glBindVertexArrayOESv; PFNGLDELETEVERTEXARRAYSOESPROC glDeleteVertexArraysOESv; glGenVertexArraysOESv = (PFNGLGENVERTEXARRAYSOESPROC)eglGetProcAddress ( "glGenVertexArraysOES" ); glBindVertexArrayOESv = (PFNGLBINDVERTEXARRAYOESPROC)eglGetProcAddress ( "glBindVertexArrayOES" ); glDeleteVertexArraysOESv = (PFNGLDELETEVERTEXARRAYSOESPROC)eglGetProcAddress ( "glDeleteVertexArraysOES" ); After these steps, the new alias glGenVertexArraysOESv, glBindVertexArrayOESv, glDeleteVertexArraysOESv can be use to call the VAO operation function in OpenGL ES 2.0 extensions.

The attached is the document and sample code for iMX5 system 80 interface LCD driver based on IPUV3. It is based on iMX51 2.6.31_09.12 BSP (SDK 1.7), tested on iMX53 3-Stack board. 1. Description This is Smartlcd driver for Freescale MX51 SDK1.7 release. (Kernel: 2.6.31_09.12.00/01)  2. File List -- Smartlcd_giantplus_4_IMX51_Linux_2.6.31_09.12.01.patch: SmartLCD panel support patch, and unit test code. -- Sample.config: the config file for reference. -- readme.txt: this file, please refer to it before use the package. -- SmartLCD Structure.pptx: the basic structure for smartlcd on IPUv3. 3. Requirement - MX51_3DS Green board(TO2.0) - No hardware rework needed, only need plug the giantplus GMA722A0 to J10. - MX51 SDK1.7 release package - L2.6.31_09.12.00_SDK_source.tar.gz                                - redboot_200952.zip 4. How to use 4.1 How to use demo -- Program default redboot.bin to board via ATKtools -- Copy attached zImage to tftp folder (assume /tftpboot) -- extract default rootfs to NFS folder (assume /nfsroot) -- COPY attached imx51_fb_test to ~/unit_test folder. -- Power on the board -- After redboot is boot up, use following command to boot up linux kernel    load -r -b 0x100000 zImage    exec -c "noinitrd console=ttymxc0 root=/dev/nfsroot rootfstype=nfsroot nfsroot=10.192.225.221:/nfsroot/rootfs rw ip=dhcp" -- Once the linux kernel launched, run following commands to test smartlcd panel.    cd /unit_tests    ./imx51_fb_test 4.2 How to use source code -- Current release code is based on L2.6.31_09.12.00_SDK_source.tar.gz. Extract the file to your working folder. -- Entering the working folder and type "./install", select a folder to install ltib. (such as .../ltib) -- Entering ltib folder and type "./ltib" to build Linux platform.  If you are not familiar with this setp, please refer to doc "i.MX_3Stack_SDK_UserGuide.pdf" for detail. -- Entering folder ".../ltib/rpm/BUILD/linux", copy "Smartlcd_giantplus_4_IMX51_Linux_2.6.31_09.12.01.patch" from release package to current folder    Run command "patch -p1 < Smartlcd_giantplus_4_IMX51_Linux_2.6.31_09.12.01.patch" -- When complete, run command "make ARCH=arm menuconfig", and you can refer to attached sample.config for detail.    * enable    Device Drivers ----> Graphics support ----> [*]   Asynchronous Panels                                            ----> [*] GiantPlus 240x320 Panel                                             * disable    Device Drivers ----> Graphics support ----> [ ]   Synchronous Panel Framebuffer                                         ----> Multimedia support    ----> [ ]   Video For Linux                                             -- Run command "make ARCH=arm" to build kernel.  4.3 How to do SMARTLCD driver test -- After Smartlcd_giantplus_4_IMX51_Linux_2.6.31_09.12.01.patch applied, there will be an folder "IMX51_TEST" under linux. -- Go to that folder, and run "make ARCH=arm", imx51_fb_test will be created. -- Copy imx51_fb_test to rootfs/unit_test. and run. 5. History N/A 6. Known Issue -- V4L2 not working yet.

This document describes the i.MX 8MM EVK mini-SAS connectors features on Linux and Android use cases, covering the supported daughter cards, the process to change Device Tree (DTS) files or Boot images, and enable these different display options on the board.

This is the prototype solution to enable second display showing different things on JB4.2.2 SabreSD. Make use of Class Presentation provided by android to be embedded into Status bar. When unlock the screen, the Presentation will show on second display. Now, the solution requires one .mp4 video placed in root sdcard. Of course, you may change it to show anything. The attached Files are a layout xml file, a patch and a recorded video. The layout file should be put into android/frameworks/base/packages/SystemUI/res/layout/ folder. The patch should be applied to frameworks/base.git. The recorded video shows the dual display demo as a reference.

gst-launch is the tool to execute GStreamer pipelines. Task Pipeline Looking at caps gst-launch -v  <gst elements> Enable log gst-launch --gst-debug=<element>:<level> gst-launch --gst-debug=videotestsrc:5 videotestsrc ! filesink location=/dev/null