Unable to play videos on QML Qt6

[imahgin]

Hi,

I've developed a QML app on an iMX93 platform (Yocto + Qt6.4) and I'm experiencing an issue when trying to play videos.
Whether I use MediaPlayer + VideoOutput or Video components, I'm not able to view anything on the screen. Audio plays ok though

This is a snippet of my code, but video QML examples also doesn't work.

    Video {
        id: video
        anchors.fill: parent
        source: "file:///home/root/turning_pages.avi"
        fillMode: Image.Stretch
        muted: false

        Component.onCompleted: play()

        onErrorOccurred: error => console.log('Error occurred:', errorString, error)
        onPlaying: console.log('playing stream', source)
        onStopped: console.log('stream stopped')
        onBufferProgressChanged: console.log('buffering:', bufferProgress)
    }

And this is the console output -no warnings nor errors-

====== AIUR: 4.7.3 build on Feb  3 2023 05:14:46. ======
	Core: AVI_PARSER_03.06.08  build on Sep 26 2022 08:17:20
 file: /usr/lib/imx-mm/parser/lib_avi_parser_arm_elinux.so.3.1
------------------------
    Track 00 [video_0] Enabled
	Duration: 0:00:15.533178000
	Language: und
    Mime:
	image/jpeg, width=(int)160, height=(int)120, framerate=(fraction)500000/33333 
------------------------
------------------------
    Track 01 [audio_0] Enabled
	Duration: 0:00:15.533381000
	Language: und
    Mime:
	audio/x-raw, format=(string)U8, channels=(int)1, layout=(string)interleaved, rate=(int)11024, bitrate=(int)88192 

I'm able to play the video using gst-launch

gst-launch-1.0 playbin uri=file:///home/root/turning_pages.avi video-sink="waylandsink sync=false"

So it seems it's not a codec related issue, but these are the gstreamer plugins installed:

CORE_IMAGE_EXTRA_INSTALL += "gstreamer1.0-plugins-ugly-meta packagegroup-fsl-gstreamer1.0-commercial"  

LICENSE_FLAGS_ACCEPTED += "commercial"  

CORE_IMAGE_EXTRA_INSTALL += " \  
 packagegroup-fsl-tools-gpu \  
 packagegroup-fsl-gstreamer1.0 \  
 packagegroup-fsl-gstreamer1.0-full \  
" 

Running Qt widgets examples does works and both video and audio works fine, so it seems only related to the QML side

When building Yocto I've also added the following lines:

IMAGE_INSTALL:append = " qtmultimedia qtmultimedia-plugins qtmultimedia-qmlplugins"

PACKAGECONFIG:append_pn-qtmultimedia = " gstreamer gstreamer010"
"

Thks in advance

[wooosaiiii]

@imahgin Can you confirm you are using i.MX93 rev=A1 + Qt6 + QML to play videos? 

[imahgin]

Yeap, but this lead to an unexpected performance issue as all animations in QML run very slowly making the app unusable.

Reverting some commits fixes this issue, but then, the Video is broken again.

Looks like switching to linuxfb -required because iMX93 has no GPU- somehow break Qt multimedia framework

[Bionicman]

Hi imahgin
Did you solve the issue with QML performance on the imx93?
I experience exactly the same issue with Qt6.8.1 and yocto-scarthgap-6.6.52_2.2.0.
QT Widgets works fine, but QML is simly very slow.

[imahgin]

Nope, with the latest BSP -scarthgap- the issue is not solved. The problem is the lack of GPU.
Applying the patches the video playback works -with very low fps- but then the QML performance is terribly slow.
So no way to use this processor to decode video.

[Bionicman]

Hi imahgin
Can you tell me exactly which partches are you talking about?
I can normally play a video using gst-play, I am not sure that the fps is low. Or maybe you refer to playing from QML?
Anyway, though there is no GPU, I find it quite strange that QML rendering itself takes so much time.

[imahgin]

We were using a Variscite module and according to them -in Micledore- the origin of the problem is in the following NXP commit
https://github.com/nxp-imx/meta-imx/commit/037aac9c51c016011c65bf3613481e5d7113ddea
You should remove these 2 line and restore the commits from
https://github.com/nxp-imx/meta-imx/pull/4/commits

These steps makes the QML be responsive again but then we loose the video decoding on QML.

There is no issues with gst-play or Qt widgets. The problem is in the QML side

[Bionicman]

I could not make it work. It seems there is something more that is not working. linuxfb is not in the list of avaialbe platforms when I run the app. The wayland platform work very slow because of QML.

[imahgin · Accepted solution]

Today I've received the latest i.MX93 SOC version and compiled again Yocto with the latest BSP, and finally the QML streaming issue is fixed!!!

So, in my case was a SOC version problem.

[imahgin]

I opened a case in NXP and the good news is that they are aware of the problem and it'll eventually be fixed in upcoming releases.
According to their response, the issue is related to libqt6-qtlocation ¿?

The problem is that new releases are only compatible with the latest i.MX93 SOC

BR

[imahgin]

I found this patch to provide Qt6 option for qmlgl.

For testing purposes, instead of creating my own layer, I directly modified gstreamer1.0-plugins-good_1.22.0.imx.bb -and also gstreamer1.0-plugins-good_1.22.0.bb just in case- with that patch

QTWAYLANDDEPENDS = "${@bb.utils.contains("DISTRO_FEATURES", "wayland", "qtwayland", "", d)}"  

PACKAGECONFIG[qt5]        = "-Dqt5=enabled,-Dqt5=disabled,qtbase qtdeclarative qtbase-native ${QTWAYLANDDEPENDS}" 
PACKAGECONFIG[qt6]        = "-Dqt6=enabled,-Dqt6=disabled,qtbase qtdeclarative qtbase-native qttools-native ${QTWAYLANDDEPENDS}" 

do_configure:prepend() { 

    # provide path to qtwaylandscanner 

    export PATH=${PATH}:${STAGING_DIR_NATIVE}/${libexecdir} 

} 

Yocto build succeeded but 2 warnings are issued

WARNING: gstreamer1.0-plugins-good-1.22.0.imx-r0 do_configure: QA Issue: gstreamer1.0-plugins-good: invalid PACKAGECONFIG: qmlglsink [invalid-packageconfig]
WARNING: gstreamer1.0-plugins-good-1.22.0.imx-r0 do_configure: QA Issue: gstreamer1.0-plugins-good: invalid PACKAGECONFIG: qmlglsrc [invalid-packageconfig]

And the plugins, obviously, are missing in the Yocto compilation.

What I'm missing here?

[imahgin]

I made some modifications to my local.conf trying to include the QML plugins:

IMAGE_INSTALL:append = " qtmultimedia qtmultimedia-plugins qtmultimedia-qmlplugins"


VIDEO_TOOLS = " \
gstreamer1.0 \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
gst-player \
gstreamer1.0-meta-base \
gst-examples \
gstreamer1.0-rtsp-server \
"
IMAGE_INSTALL:append = " ${VIDEO_TOOLS}"
LICENSE_FLAGS_ACCEPTED += "commercial"

PACKAGECONFIG:append:pn-gstreamer1.0-plugins-good = " qmlglsink"
PACKAGECONFIG:append:pn-gstreamer1.0-plugins-good = " qmlglsrc"
PACKAGECONFIG:append:pn-qtmultimedia = " gstreamer qml"

But again I have no QML plugins installed when building Yocto.

How can I instruct bitbake to include gstreamer QML plugins in my compilation?

[imahgin]

According to the i.MX Linux Release Notes Rev. LF6.1.22_2.0.0, Gstreamer has been upgraded to 1.22.0!!!
I've built a new Yocto image based on mx93-yocto-mickledore-6.1.22_2.0.0-v1.0 and checked that gstreamer version is really 1.22.0, but still cannot play video -sound works fine- and no errors are displayed on console.

[imahgin]

According to this information seems that Qt6 support for rendering video inside a QML scene is available only from version 1.22.0, and according to this document this version is included in Rev. LF6.1.36_2.1.0 (page 9)

My distribution has 1.20.3. As I'm not an experienced user in Yocto, is there any 'easy' way to include this -or latest version- on our current Yocto compilation, without breaking other packages?

Thks

[imahgin]

Some updates from my part.

I've compiled a Dunfell distribution -based on Qt5- and everything works fine out of the box.

Didn't install any additional codec and video plays fine from both command line and QML app.

It looks like it's only related to the compiled Multimedia component on embedded Qt6

Any other clue?

[brian14]

Hi @imahgin, 

Thank you for contacting NXP Support.

You can try using the Gstreamer pipeline in your QML code.

Here is an example:

Gstreamer in command line pipeline:

$ gst-launch-1.0 -e filesrc location=video.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! waylandsink

Gstreamer pipeline used in QML code:

source : "gst-pipeline: filesrc location=/home/root/video.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! qtvideosink"

I hope this information will helpful.

[imahgin]

Hi @brian14

Thks for the suggestion but I've already tried a pipeline (using qmlglsink on avi file), but neither of them works in QML

The following message is displayed on console:

qt.multimedia.player: Unable to set the pipeline to the paused state.

If running gst-launch, then some components are missed

erroneous pipeline: no element "v4l2h264dec"
erroneous pipeline: no element "imxvideoconvert_g2d"
erroneous pipeline: could not link queue0 to waylandsink0

but the following do works:

gst-launch-1.0 playbin uri=file:///home/root/video.mp4 video-sink="waylandsink sync=false"

so I guess some codecs are missed in my Yocto project.

I had no issues using Qt5 without using pipelines, so I'll dig deeper into my Yocto compilation.