Poor video playback on i.MX6 using QtMultimedia 5.5.1

[jasonwright]

I'm trying to get a video playing in my Qt5.5.1 app. I'm using an i.MX6D board (VAR-SOM-MX6) with Yocto Jethro and X11 graphics, with a few additional things in my local.conf to get gstreamer working:

PACKAGECONFIG_append = " packagegroup-fsl-gstreamer" 
PACKAGECONFIG_append_pn-qtmultimedia = " gstreamer" 
IMAGE_INSTALL_append = " libnl libnl-genl qtbase-plugins qtmultimedia-plugins qtimageformats-plugins gstreamer gst-meta-video gst-plugins-base-app gst-plugins-base gst-plugins-gooimx-vpu imx-test firmware-imx-vpu-imx6d" 

In my Qt app, I created a QVideoWidget (called video_widget) and loaded the video using the following code:

QMediaPlayer * player = new QMediaPlayer(this);
player->setMedia(QUrl::fromLocalFile("/home/root/test.m4v"));
player->setVideoOutput(ui->video_widget);
player->play();

The video plays, but performance is very bad (<5 FPS) compared to just directly using gstreamer. I also get a slew of warnings each time, along the lines of: 

QWidget::paintEngine: Should no longer be called
QWidget::paintEngine: Should no longer be called
QWidget::paintEngine: Should no longer be called
QWidget::paintEngine: Should no longer be called
QWidget::paintEngine: Should no longer be called
QWidget::paintEngine: Should no longer be called
Warning: "A lot of buffers are being dropped."

I have tried this with a couple different codecs and a couple different videos, but am primarily using h264-encoded videos in .m4v containers, since that seemed to yield the best result. Changing the video dimensions and bitrate didn't seem to have any effect.

As a comparison:

https://vimeo.com/183730687 -- this is a video being played using QtMultimedia.

https://vimeo.com/183731187 -- this is the same video played by running

gst-launch-1.0 playbin uri=file:/home/root/test.m4v

Does anyone have any experience with this... or any suggestions?

[Sun1z]

So many years passed, it seems this issue is still exist in recent 5.4.70_2.3.0 GA. ^_^

for qml application, as there is a 'qmlsink', so the application will take the advantage of the codec, although it's a bit slower than gst-play.

for qwidget application, it seems there is no hw codec, so the performance is very poor.

[igorpadykov]

Hi Jason

there are two gstreamer versions as described on

trac.gateworks.com/wiki/Yocto/gstreamer

One can try both, nxp gstreamer plugins are described in attached

sect.7 Multimedia, sect.8.4 Qt 5 attached Linux Guide with nxp bsps on

i.MX 6 Series Software and Development Tool|NXP 

To check if this is caused by some hardware limitations

one can try imx-test video tests (/mxc_vpu_test)

www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-test-5.4.tar.gz

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[jasonwright]

Thanks Igor--

I used gstramer1.0 instead of gstreamer0.1, because building Yocto with 

PACKAGECONFIG_append_pn-qtmultimedia = " gstreamer010"
yielded these series of errors when running the Qt app:

(gst-plugin-scanner:1194): GLib-GObject-WARNING **: cannot register existing type 'GstVorbisDec'
(gst-plugin-scanner:1194): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed
(gst-plugin-scanner:1194): GStreamer-CRITICAL **: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
Warning: "No decoder available for type 'audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)2, rate=(int)44100, channels=(int)2, parsed=(boolean)true'."
Warning: "No decoder available for type 'video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3, profile=(string)high, codec_data=(buffer)0164001effe1001d6764001eacd9408837ee7ffc0070006c400000fa40003a9823c58b658001000668ebe3cb22c0, width=(int)540, height=(int)420, framerate=(fraction)30000/1001, pixel-aspect-ratio=(fraction)28/27, parsed=(boolean)true'."
Error: "Your GStreamer installation is missing a plug-in."
Warning: "No decoder available for type 'audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)2, rate=(int)44100, channels=(int)2, parsed=(boolean)true'."
Warning: "No decoder available for type 'video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3, profile=(string)high, codec_data=(buffer)0164001effe1001d6764001eacd9408837ee7ffc0070006c400000fa40003a9823c58b658001000668ebe3cb22c0, width=(int)540, height=(int)420, framerate=(fraction)30000/1001, pixel-aspect-ratio=(fraction)28/27, parsed=(boolean)true'."
Error: "Your GStreamer installation is missing a plug-in."

Since the video plays fine using gst-launch, I don't think it could be a hardware limitation.