This post shows a workaround about how to embed video on your QT Qml application with high performance on i.MX8MPlus.  The demo works on Linux BSP L6.1.22_2.0.0 and weston branch weston-imx-11.0. This demo set the glimagesink pipeline on weston top view and  works for 1080p screen.  QT MediaPlayer Since qmlglsink is not supported now on QT6, one way we can display video with QML application is to use MediaPlayer component. However, it has bad performance when playing with high resolution videos. From my test, while playing with 1080p60 video, cpu loading is around 50% and some frames will be lost.  With pure Gstreamer pipeline,  gst-launch-1.0 -vvv filesrc location=3.mp4 ! qtdemux  ! vpudec ! video/x-raw,format=NV12 ! imxvideoconvert_g2d ! queue  ! glimagesink It works good with 1080p60 video.   Therefore, we could make it work with QML application to achieve the best performance on our platform.   Demo explanation  The demos works with QT6 qml  and the Gstreamer pipeline is gst_parse_launch("filesrc location=3.mp4 ! qtdemux ! vpudec ! video/x-raw,format=NV12 ! imxvideoconvert_g2d ! video/x-raw,format=BGRx,width=1280,height=720 ! queue ! glimagesink ",NULL);   You can change 3.mp4 to your video.  if you want to change the size of glimagesink output , you need also change the QML rectangle and weston.ini.   Compile Steps Weston git clone https://github.com/nxp-imx/weston-imx.git git checkout weston-imx-11.0.1 git apply 0001_Weston_Fix_Glimagesink_Pos_and_QML_APP.patch   Follow the below link to compile weston-11 and copy them to the rootfs. Weston-imx intergration guide - Graphics Technology Engineering Center - NXP Confluence   QML application source toolchain cmake . make   In CMakeLists.txt, you need to modify the INCLUDE_DIRECTORIES to your toolchain path.     Run Steps Add the following component to the shell section on the /etc/xdg/weston/weston.ini qtapp is the name of your app. qt_pos sets the initial x and y position of your app. glsink_pos sets the initial x and y position of your gstreamer glimagesink pipeline. These parameters can set your gstreamer video on your application rectangle for playing video, test and choose for your best.  systemctl restart weston ./FIXVIDEO Result           You can see your Gstreamer video is always on top and you can control your pipeline by press the Start, Pause and Stop button.      Pending to do:                This is just a simple demo for 1080p screen and shows the video with 1280x720 rectangle. If your screen and show-rectangle is different, then you need to modify not only the qml file, the Gstreamer pipeline, but also the weston.ini to fit your video on the rectangle.          The video is fixed on the position you set and can not be moved.       Reference Link  i.MX8MPlus :https://www.nxp.com/design/design-center/development-boards/i-mx-evaluation-and-development-boards/e...  weston: https://github.com/nxp-imx/weston-imx     

This brief training covers the basics on how Weston works. It includes sample code for the user to test and learn about the basics.

i.MX Graphics User's Guide   GPU Driver Releases Description GPU Driver Graphics Tools BSP Supported GPU-VIV 5.0.11 P8 imx-gpu-viv-5.0.11.p8.1-hfp.bin imx-gpu-viv-5.0.11.p8.1-sfp.bin kernel-module-imx-gpu-viv-5.0.11.p8.1.tar.gz imx-gpu-viv-5.0.11.p8.1.tgz Linux: L4.1.15 1.0.0 Android: Android 6.0 GPU-VIV 5.0.11P7 imx-gpu-viv-5.0.11.p7.4-hfp.bin imx-gpu-viv-5.0.11.p7.4-sfp.bin kernel-module-imx-gpu-viv-5.0.11.p7.4.tar.gz L3.14.52_1.1.0-ga_gpu-tools.tar.gz Linux: L3.14.52 1.1.0 GPU-VIV 5.0.11P7 imx-gpu-viv-5.0.11.p7.1-hfp.bin imx-gpu-viv-5.0.11.p7.1-sfp.bin No separate kernel driver package. Included as part of BSP L3.14.52_1.1.0-ga_gpu-tools.tar.gz Linux: L3.14.38 1.0.0 GPU-VIV 5.0.11 P4 imx-gpu-viv-5.0.11.p4.1-hfp.bin imx-gpu-viv-5.0.11.p4.1-sfp.bin No separate kernel driver package. Included as part of BSP i.MX6D/Q/DL/S/SX GPU Tools-5.0.11P4.1 Linux: L3.10.53, L3.14.28 GPU-VIV 4.6.9 P13 gpu-viv-bin-mx6q-3.10.17-1.0.3-hfp.bin, gpu-viv-bin-mx6q-3.10.17-1.0.3-sfp.bin No separate kernel driver package. Included as part of BSP I.MX_6_VIVANTE_VDK_150_TOOLS Linux: L3.10.17, L3.0.101

This post shows a new feature (weston_clone_mode) that many customers need before in i.MX8MPlus.  The demo works on Linux BSP L5.10.72_2.2.0 and weston branch weston-imx-9.0. The detailed introduction, running steps and test result are shown in the attached PPT. Default Extend Mode Pipeline:   Clone Mode Pipeline:   The DSI buffers which as the clone source in the pipeline can also be changed to HDMI or LVDS buffers. Running Steps: git apply  clone_difResolution.diff source /opt/fsl-imx-xwayland/5.10-hardknott/environment-setup-cortexa53-crypto-poky-linux meson build/ --prefix=/usr -Ddoc=false -Dbackend-drm-screencast-vaapi=false -Dcolor-management-lcms=false -Dcolor-management-colord=false -Dpipewire=false -Dbackend-x11=false -Dbackend-fbdev=false -Dxwayland=true -Drenderer-g2d=true DESTDIR=./opt/weston ninja -C build/ install tar -cvf weston_usr.tar build/opt/weston/usr/* copy the weston_usr.tar to your evk board systemctl stop Weston tar -xvf weston_usr.tar cp -r  build/opt/weston/* / & sync Adding “clone-mode=1”, “clone-connector-id=41” to your /etc/xdg/weston/weston.ini “core” section like picture (1). You can also add “cloneoutput” section like picture (2)  If you decides to use other modes of your screen. systemctl restart Weston 41 or 45 in the pictures are the connector id of the corresponding screens. You can use modetest –M imx-drm to see the connector id.     Test Results: Three displays: Two Displays:    Reference Link  i.MX8MPlus :https://www.nxp.com/design/design-center/development-boards/i-mx-evaluation-and-development-boards/evaluation-kit-for-the-i-mx-8m-plus-applications-processor:8MPLUSLPD4-EVK  weston: https://github.com/nxp-imx/weston-imx  weston doc: https://wayland.pages.freedesktop.org/weston/      HPD update   Please update the patch to clone_difResolution_add_HPD.diff which supports HPD feature. Notion: pleas add force-on=true to the output you use as main screen. For example.            

This is only a test of keep u-boot logo util  yocto system screen on for some request validation. HW : 8mn-ddr4-evk. SW:  8mn 4.14.98ga-2.2.0 release. 1. patch 2 u-boot patchs. 2. patch kernel patch. 3. changed kernel config to remove  CONFIG_FRAMEBUFFER_CONSOLE and  CONFIG_LOGO.

Find attached an introductory course to the NXP GPU Demo Framework.   The sample Projects will be uploaded separately.   Cheers, Hugo

<<TBD>> gpuinfo, gmem_info, vProfiler, apitrace, mmdc profiler, top, provide version info for OS, driver, window manager, HMI framework

FSL G2D API on top of Vivante GPU 2D API

i.MX6 GPU memory tool by Xianzhong Li

  This post shows a way to use the g2d physical contiguous memory for the ISI to using userptr, this buffer will handled by openCL directly, without memcpy. This demo works on: i.MX8MP L6.1.36, It should be noted that only ISI can support userptr, ISP does not support userptr. In NXP default BSP release, the camera application only support mmap, It maps buffers from kernel space to the user space, you need to memcpy these buffers for future use:   We can make some change to use the userptr to avoid this memcpy, the userptr, as the name implies, refers to the user space pointer. the application layer allocates the required memory space and passes it to the v4l2 driver layer as a pointer, the v4l2 driver saves the content of capture to the space indicated by the pointer.     Solution   1. Firstly, ISI driver removed userptr to avoid non-continuous memory, we need to revert this change first: LF-3334-06: media: isi: remove userptr io mode for ISI capture driver · nxp-imx/linux-imx@10b125f · GitHub 2.  Revert get_vaddr_frames function, patch as attached. 3. Change your application, sample application is attached, it creates g2d physical contiguous memory with g2d_alloc, and set uncached memory with this flag CL_MEM_USE_UNCACHED_HOST_MEMORY_VIV.