i.MX6 XWayland QT Slow Video Playback

[ericholmberg]

I am in the process of upgrading an older i.MX6 QT over X11 system to the latest NXP Yocto Release 5.10.9_1.0.0.

The system is configured for XWayland for the imx6qsabresd.  XWayland was chosen to allow 32-bit QT applications to run on XWayland on both iMX6 and iMX8 hardware.

A simple QT Video playback test program plays the 24 FPS video at around 1 FPS.  Using gst-play-1.0 or telling QT to use the iMX Video compositor sink runs at the expected 24 FPS, so the problem is in the QT video overlay sink or compositing process.

Does anybody have any ideas as to what is missing in QT?

The system is configured and built as the default imx6qsabresd system:

DISTRO=fsl-imx-xwayland MACHINE=imx6qsabresd source imx-setup-release.sh -b build-xwayland

Using gst-play works fine along with building a basic playback pipeline using gst-launch.

# overlaysink and imxv4l2sink both work fine at full 24 FPS
gst-launch-1.0 filesrc location=/home/vendor/videos/coke.mp4 typefind=true ! qtdemux ! queue
! vpudec ! imxv4l2sink overlay-width=320 overlay-height=240 force-aspect-ratio=1

gst-launch-1.0 filesrc location=/home/vendor/videos/coke.mp4 typefind=true ! qtdemux ! queue
! vpudec ! overlaysink overlay-width=320 overlay-height=240 force-aspect-ratio=1

The QT playback pipeline uses QGstVideoRendererSink by default.  If that is changed to either the overlaysink or imxv4l2sink, then video is rendered at full 24 FPS, but the video size is not matched to the QVideoWindow.

export QT_GSTREAMER_WINDOW_VIDEOSINK=overlaysink
./testplay

export QT_GSTREAMER_WINDOW_VIDEOSINK=imxv4l2sink
./testplay

QT Configuration

Qt Multimedia:
  ALSA ................................... yes
  GStreamer 1.0 .......................... yes
  GStreamer 0.10 ......................... no
  Video for Linux ........................ yes
  OpenAL ................................. no
  PulseAudio ............................. yes
  Resource Policy (libresourceqt5) ....... no
  Windows Audio Services ................. no
  DirectShow ............................. no
  Windows Media Foundation ............... no


Qt is now configured for building. Just run 'make'.
Once everything is built, you must run 'make install'.

QT Test Application

// QT Test Code
int main(int argc, const char* argv[])
{
QApplication app(argc, const_cast<char **>(argv));

QWidget window;
window.resize(QApplication::desktop()->screenGeometry().width(), QApplication::desktop()->screenGeometry().height());
window.setWindowState(Qt::WindowFullScreen);
window.show();

QMediaPlayer player;
QVideoWidget video(&window);

video.resize(320, 240);
player.setVideoOutput(&video);

player.setMedia(QUrl::fromLocalFile("/home/test/videos/test-video.mp4"));
player.play();
video.show();

 return app.exec();
}

[ericholmberg]

[deleted and reposted as reply to Bio_TICFSL

[ericholmberg]

I had a look through gst-tools and found the issue.  QT is configured for Wayland and QGstreamerVideoOverlay does not have Wayland support, so it sees that the platform is not X and falls back to the software renderer QGstVideoRendererSink.  The iMX6 can't keep up hence the slow framerate.

The platform is configured as XWayland, so the XCB interface should work.  I'll look into how to configure QT to use X in this case.  Any suggestions are appreciated.

[ericholmberg]

When telling QT to use X11 by passing in the -platform xcb option, the application prints an error message and crashes about 30 seconds later.

export DISPLAY=:0
./testplay -platform xcb
QXcbIntegration: Cannot create platform OpenGL context, neither GLX nor EGL are enabled
display(/dev/fb0) resolution is (640x480).

Debugging the issue shows that the crash is caused by ximagesink.  Commenting that out in qgstreamervideooverlay.cpp allows video playback to work the same as when using gst-lauch-1.0, but the window size is not being configured.

From:  gsttools/qgstreamervideooverlay.cpp

// Ordered by descending priority
static const ElementMap elementMap[] =
{
#if QT_CONFIG(gstreamer_gl)
{ "xcb", "glimagesink" }, // Cannot create platform OpenGL context, neither GLX nor EGL are enabled
#endif
// { "xcb", "xvimagesink" }, // not used
// { "xcb", "ximagesink" }, // doesn't play and results in OOM after 20 seconds.
{ "xcb", "imxv4l2sink" }, // works, but not sized to window
{ "xcb", "overlaysink" }, // works, but not sized to window
};

I will look into setting the window size properties, but would like to know if other people have been able to get this working.

[Bio_TICFSL]

Hello ericholmberg,

My guess is that the Qt application is trying to perform some color space conversion in SW. You might want to extract the gstreamer pipeline generated by the application (GST_DEBUG_DUMP_DOT_DIR ?) and verify what is happening there

Regards

[ericholmberg]

Reposted as reply in the correct spot.

Yes, that's a good idea.  I pulled the pipeline as part of the early debug and started to simplify down the massive pipeline.  The gst-play pipeline that works does what looks like a pass-through GstPlaySinkVideoConvert from NV12 to NV12.

The slow playback through QT shows a GstPlaySinkVideoConvert from NV12 to BGRx which may confirm what you were suspecting.

Once I added the imxv4l2sink video sink to QGstreamerVideoOverlay which fixes the performance issues, the piepline dump is simple, but essentially useless -- not sure why.

I'll share the patches in a follow-up post in case other people run into the same issue.