Installing GStreamer on I.MX8QM

[mustafa_ozcelik]

Hi all,

I am trying to figure out how to properly install gstreamer on imx8qm-mek (for a qt app to use).
I am using the morty branch (beta2 release) for imx8qm.
More precisely Linux L4.9.51 for i.MX 8QuadMax Beta2 and i.MX 8QXPlus Beta  from Alpha / Beta BSPs for Microprocessors|NXP . And I am building an image with weston/wayland.
I have the following in my image recipe:

IMAGE_INSTALL_append = " qtmultimedia \

gstreamer1.0 \

gstreamer1.0-plugins-base \

gstreamer1.0-plugins-bad \

packagegroup-gsl-gstreamer1.0 \

gstreamer1.0-plugins-base-meta \

gstreamer1.0-plugins-bad-meta \
gstreamer1.0-plugins-good-meta \
libnl \

libnl-genl \

qtbase-plugins \

qtmultimedia-plugins \

qtimageformats-plugins \

"

When I create the image, qt media player app (which works in Raspberry Pi) complains:

====== BEEP: 4.3.3 build on Aug 13 2018 18:58:20. ======

Core: AAC decoder Wrapper build on Dec 7 2017 18:13:51

file: /usr/lib/imx-mm/audio-codec/wrap/lib_aacd_wrap_arm_elinux.so.3

Warning: "No decoder available for type 'video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)avc, width=(int)1280, height=(int)720, framerate=(fraction)25/1, codec_data=(buffer)014d401fffe1001c674d401fe8802802dd80b501010140000003004000000c83c60c448001000468ebef20, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, profile=(string)main, level=(string)3.1'."

CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.09.00_ARMV8 build on Sep 20 2017 15:02:50.

====== BEEP: 4.3.3 build on Aug 13 2018 18:58:20. ======

Core: MP3 decoder Wrapper build on Dec 7 2017 18:14:28

file: /usr/lib/imx-mm/audio-codec/wrap/lib_mp3d_wrap_arm_elinux.so.3

CODEC: BLN_MAD-MMCODECS_MP3D_ARM_02.13.00_ARMV8 build on Feb 13 2017 19:47:36.

====== AIUR: 4.3.3 build on Aug 13 2018 18:58:16. ======

Core: MPEG4PARSER_06.12.12 build on Oct 27 2017 02:58:02

file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm_elinux.so.3.2

------------------------

Track 00 [video_0] Enabled

Duration: 0:03:58.600000000

Language: und

Mime:

video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)avc, width=(int)1280, height=(int)720, framerate=(fraction)25/1, codec_data=(buffer)014d401fffe1001c674d401fe8802802dd80b501010140000003004000000c83c60c448001000468ebef20

------------------------

------------------------

Track 01 [audio_0] Enabled

Duration: 0:03:58.608244000

Language: eng

Mime:

audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)44100, bitrate=(int)0, stream-format=(string)raw, codec_data=(buffer)1210

------------------------

Furthermore, I get the following (which I was able to get rid off by removing gst- plugins (not the gstreamer1.0 ones)

(gst-plugin-scanner:9753): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-1.0/libgstqt5videosink.so': libXdamage.so.1: cannot open shared object file: No such file or directory

Now, I noticed that in the master branch of meta-freescale, there are some new recipes under gstreamer folder, namely: gstreamer1.0-plugins-imx gstreamer1.0_1.12.imx. I am unable to integrate these because of the syntax changes in Yocto from morty to rocko. Are these packages important. Is it sufficient to have morty release for a complete gstreamer install?

Any suggestions are highly appreciated.

[joanxie]

it seems you haven't add vpu decoder in the bsp, one can try to install libav (software decoder).

The following steps show how to install gstreamer1.0-libav into a rootfs image.
1. Add the following lines into the configuration file conf/local.conf.
IMAGE_INSTALL_append = " gstreamer1.0-libav"
LICENSE_FLAGS_WHITELIST = "commercial"
2. Build gstreamer1.0-libav.
$ bitbake gstreamer1.0-libav
3. Build the rootfs image.
$ bitbake
$ <image_name>

[mustafa_ozcelik]

Hi,
I tried this today without nowing it is the software decoder.
There is some improvement, yes. Now I can at least see the player work, however there is 80% of the time black screen. 20% of the time I see the flickering images of the movie.

Can I assume that is it because the software decoder isn't fully supported efficiently?
Is it maybe because I have to include other decoders such as AAC+ etc from external repositories?

I think i.MX8QM does not fully support multimedia related things just yet.

The following is from release notes:

6.4 Known issues and limitations for multimedia
Issues seen on GStreamer 1.x:
• As the maximum buffer size of the playbin multiqueue is 2 MB, problems may be seen with some long audio or video
interleaved streams. You can enlarge this buffer size to support these special use cases.
• AAC decoder: The ADIF format does not support seek mode nor FF/FB.
• The accurate seek mode may have a longer time delay.
• Because the stream container does not have an index table, seeking is not supported.
• Fast rewind of audio does not support audio-only streams.
• Pulseaudio is only available for the X11 backend.
• Rewind may report an EOS when using libav for video decoding.
•
•
•
•
•
•
For the clips with long audio/video interleaved streams, it requires to enlarge the multiqueue maximum buffer size.
AAC clips with ADIF format do not support seek/trick (FB/FF) mode.
Accurate seek mode may take a longer time delay.
For clips with no index table in containers (or video with only very few key frames), seeking is not supported.
Audio-only clips do not support FB.
For pulseaudio, it automatically starts with X11 backend. For others, users need to manually start the daemon.

What else could I try?
Thank you very much for the information

[mustafa_ozcelik]

Okay, now I tried using gst-play-1.0, it works.
My Qt app has something to do with the current issue. Do you know any methods I could try to make QtMultimedia use this correctly?

Full image recipe is below:

#Sound related
DISTRO_FEATURES_append = " pulseaudio "

IMAGE_INSTALL_append = "  alsa-lib \
                          alsa-tools \
                          alsa-state \
                          pulseaudio \
                          imx-alsa-plugins \
"

# Qt dependencies for infotainment application
IMAGE_INSTALL_append = " qtconnectivity qtquickcontrols2 \
                         qtvirtualkeyboard qtwebsockets \
                         qtmultimedia     qtbase-tools \
                         qtbase-plugins \
                         qtdeclarative \
                         qtdeclarative-tools \
                         qtdeclarative-qmlplugins \
                         qtmultimedia \
                         qtmultimedia-plugins \
                         qtmultimedia-qmlplugins \
                         qtsvg \
                         qtsvg-plugins \
"

# Gstreamer and its plugins
# gstreamer1.0-plugins-imx is not compatible with imx8 at the moment! Add it also..
#IMAGE_INSTALL_append = " imx-vpu imx-vpuwrap   "
IMAGE_INSTALL_append = " imx-codec imx-parser v4l-utils imx-gst1.0-plugin imx-alsa-plugins"
#IMAGE_INSTALL_append = " imx-aacpcodec imx-mscodec imx-msparser imx-ac3codec imx-ddpcodec imx-real imx-hifi4 imx-hifi4-aacp "

IMAGE_INSTALL_append = " gstreamer1.0 \
                         gstreamer1.0-plugins-base \
                         gstreamer1.0-plugins-good \
                         gstreamer1.0-plugins-bad \
"

IMAGE_INSTALL_append = " packagegroup-fsl-gstreamer1.0 gstreamer1.0-plugins-base-meta  gstreamer1.0-plugins-good-meta gstreamer1.0-plugins-bad-meta "

IMAGE_INSTALL_append = " gstreamer1.0-libav ffmpeg"

#IMAGE_INSTALL_append = " gstreamer1.0-omx "

IMAGE_INSTALL_append = " libnl libnl-genl \
                         qtbase-plugins \
                         qtmultimedia-plugins \
                         qtimageformats-plugins  \
                         qtmultimedia-qmlplugins \
"

IMAGE_INSTALL_append = " packagegroup-fsl-tools-testapps "

# Geoclue
IMAGE_INSTALL_append = " geoclue "

Thank you very much again.

[dan_searles]

Hello,

Could you give name, location and any needed references for the above mentioned "Full image recipe"?

Thanks

[joanxie]

what is qt app issue? if you play the AAC+ file, you need the special codec package which is under NDA.

[mustafa_ozcelik]

Sorry for confusing question. I have a problem using GStreamer + QtMultimedia on iMX8. This might be out of scope for this forum, but here is the question if anybody would like to help.

qt - Asking QtMultimedia to use different video sink using QML - Stack Overflow 

Thanks

[joanxie]

one can use qtmultimdia(qlmvideo) to playback video by VPU, try to add gstreamer into qtmultimeda like this:

PACKAGECONFIG_append_pn-qtmultimedia = "gstreamer" in the lcoal.conf