i.MX6 XWayland QT Slow Video Playback

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

i.MX6 XWayland QT Slow Video Playback

2,438 Views
ericholmberg
Contributor III

I am in the process of upgrading an older i.MX6 QT over X11 system to the latest NXP Yocto Release 5.10.9_1.0.0.

The system is configured for XWayland for the imx6qsabresd.  XWayland was chosen to allow 32-bit QT applications to run on XWayland on both iMX6 and iMX8 hardware.

A simple QT Video playback test program plays the 24 FPS video at around 1 FPS.  Using gst-play-1.0 or telling QT to use the iMX Video compositor sink runs at the expected 24 FPS, so the problem is in the QT video overlay sink or compositing process.

Does anybody have any ideas as to what is missing in QT?

The system is configured and built as the default imx6qsabresd system:

DISTRO=fsl-imx-xwayland MACHINE=imx6qsabresd source imx-setup-release.sh -b build-xwayland

 

Using gst-play works fine along with building a basic playback pipeline using gst-launch.

# overlaysink and imxv4l2sink both work fine at full 24 FPS
gst-launch-1.0 filesrc location=/home/vendor/videos/coke.mp4 typefind=true ! qtdemux ! queue
! vpudec ! imxv4l2sink overlay-width=320 overlay-height=240 force-aspect-ratio=1

gst-launch-1.0 filesrc location=/home/vendor/videos/coke.mp4 typefind=true ! qtdemux ! queue
! vpudec ! overlaysink overlay-width=320 overlay-height=240 force-aspect-ratio=1

 

The QT playback pipeline uses QGstVideoRendererSink by default.  If that is changed to either the overlaysink or imxv4l2sink, then video is rendered at full 24 FPS, but the video size is not matched to the QVideoWindow.

export QT_GSTREAMER_WINDOW_VIDEOSINK=overlaysink
./testplay

export QT_GSTREAMER_WINDOW_VIDEOSINK=imxv4l2sink
./testplay

 

QT Configuration

Qt Multimedia:
  ALSA ................................... yes
  GStreamer 1.0 .......................... yes
  GStreamer 0.10 ......................... no
  Video for Linux ........................ yes
  OpenAL ................................. no
  PulseAudio ............................. yes
  Resource Policy (libresourceqt5) ....... no
  Windows Audio Services ................. no
  DirectShow ............................. no
  Windows Media Foundation ............... no


Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.

 

QT Test Application

// QT Test Code
int main(int argc, const char* argv[])
{
QApplication app(argc, const_cast<char **>(argv));

QWidget window;
window.resize(QApplication::desktop()->screenGeometry().width(), QApplication::desktop()->screenGeometry().height());
window.setWindowState(Qt::WindowFullScreen);
window.show();

QMediaPlayer player;
QVideoWidget video(&window);

video.resize(320, 240);
player.setVideoOutput(&video);

player.setMedia(QUrl::fromLocalFile("/home/test/videos/test-video.mp4"));
player.play();
video.show();

return app.exec();
}

 

0 Kudos
5 Replies

2,405 Views
ericholmberg
Contributor III

[deleted and reposted as reply to Bio_TICFSL

0 Kudos

2,428 Views
ericholmberg
Contributor III

I had a look through gst-tools and found the issue.  QT is configured for Wayland and QGstreamerVideoOverlay does not have Wayland support, so it sees that the platform is not X and falls back to the software renderer QGstVideoRendererSink.  The iMX6 can't keep up hence the slow framerate.

The platform is configured as XWayland, so the XCB interface should work.  I'll look into how to configure QT to use X in this case.  Any suggestions are appreciated.

0 Kudos

2,424 Views
ericholmberg
Contributor III

When telling QT to use X11 by passing in the -platform xcb option, the application prints an error message and crashes about 30 seconds later.

export DISPLAY=:0
./testplay -platform xcb
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
display(/dev/fb0) resolution is (640x480).

Debugging the issue shows that the crash is caused by ximagesink.  Commenting that out in qgstreamervideooverlay.cpp allows video playback to work the same as when using gst-lauch-1.0, but the window size is not being configured.

From:  gsttools/qgstreamervideooverlay.cpp

// Ordered by descending priority
static const ElementMap elementMap[] =
{
#if QT_CONFIG(gstreamer_gl)
{ "xcb", "glimagesink" }, // Cannot create platform OpenGL context, neither GLX nor EGL are enabled
#endif
// { "xcb", "xvimagesink" }, // not used
// { "xcb", "ximagesink" }, // doesn't play and results in OOM after 20 seconds.
{ "xcb", "imxv4l2sink" }, // works, but not sized to window
{ "xcb", "overlaysink" }, // works, but not sized to window
};

 

I will look into setting the window size properties, but would like to know if other people have been able to get this working.

0 Kudos

2,419 Views
Bio_TICFSL
NXP TechSupport
NXP TechSupport

Hello ericholmberg,

My guess is that the Qt application is trying to perform some color space conversion in SW. You might want to extract the gstreamer pipeline generated by the application (GST_DEBUG_DUMP_DOT_DIR ?) and verify what is happening there

 

Regards

 

0 Kudos

2,404 Views
ericholmberg
Contributor III

Reposted as reply in the correct spot.

Yes, that's a good idea.  I pulled the pipeline as part of the early debug and started to simplify down the massive pipeline.  The gst-play pipeline that works does what looks like a pass-through GstPlaySinkVideoConvert from NV12 to NV12.

gst-play-pipeline-ok.png

The slow playback through QT shows a GstPlaySinkVideoConvert from NV12 to BGRx which may confirm what you were suspecting.

qt-play-slow.png

Once I added the imxv4l2sink video sink to QGstreamerVideoOverlay which fixes the performance issues, the piepline dump is simple, but essentially useless -- not sure why.

qt-play-fixed.png

I'll share the patches in a follow-up post in case other people run into the same issue.

0 Kudos