Yocto GStreamer Video Problem [fsl-image-fb]

sumanranjan · ‎04-14-2014

Board : imx6sabresd quadcore board

Recipe : fsl-image-fb

Step 1: get the code

$ mkdir fsl-community-bsp

$ cd fsl-community-bsp

$ repo init -u http://github.com/Freescale/fsl-community-bsp-platform -b dora

$ mv .repo/manifest.xml .repo/manifest.orig #remove community manifest

$ cp <location>/fsl-yocto-release-manifest.xml .repo/manifest.xml #replace with freescale manifest.xml within Yocto Release

$ repo sync #sync the repo

$ cp -r sources/meta-fsl-bsp-release/imx/meta-fsl-arm/conf/machine sources/metafsl-arm/conf # This step copies the new machine configuration files into the meta-fsl-arm so that

# community setup can find new machine files.

Step 2: build it

$ MACHINE=imx6qsabresd source setup-environment build-fb

$ cd ..

$ source fsl-setup-release.sh -b build-fb -e fb

$ bitbake fsl-image-fb

Things build OK and graphics OpenGL/ES tests run fine on imx6sabresd board. However, GStreamer has a problem in this image. First of all, I see that only few GStreamer plugins are copied to filesystem rootfs, but many plugins are present in 'sysroot' folder. Why this is so ??

1. gst-inspect command shows very few plugins.

Total count: 4 plugins, 26 features

When I checked in the "sysroot" of the build folder I found many plugins were present : ../fsl-community-bsp/build-fb/tmp/sysroots/imx6qsabresd/usr/lib/gstreamer-0.10. I copied all plugins from sysroot folder

to rootfs folder ( /usr/lib/).

After this gst-inspect shows many plugins.

Total count: 177 plugins (1 blacklist entry not shown), 594 features

After copying plugins from sysroot folder below tests pass.

a. .264 file is playing successfully.

gst-launch filesrc location=FVDO_Plane_4cif.264 typefind=true ! vpudec ! mfw_v4lsink

b. rtsp camera stream is decoded successfully.

gst-launch rtspsrc location=rtsp://10.20.1.48/axis-media/media.amp typefind=true ! gstrtpjitterbuffer ! rtph264depay queue-delay=0 ! capsfilter caps="video/x-h264,width=640,height=480,framerate=(fraction)30/1" ! vpudec low-latency=true ! queue max-size-buffers=0 max-size-time=0 ! mfw_v4lsink sync=false

2. Before copying plugins from sysroot 'aiurdemux' was not even present, but after copy 'aiurdemux' shows up in 'gst-inspect' command and throws following error.

# gst-inspect aiurdemux

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_element_class_add_pad_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_new_from_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_activate_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_activatepull_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_activatepush_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_chain_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_event_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_element_add_pad: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_setcaps_function: assertion `GST_IS_PAD (pad)' failed

Factory Details:

Long name: aiur universal demuxer

Class: Codec/Demuxer

Description: demux container file to video, audio, and subtitle

Author(s): Multimedia Team <shmmmw@freescale.com>

Rank: primary + 1 (257)

Plugin Details:

Name: aiur.imx

Description: aiur universal demux/mux

Filename: /usr/lib/gstreamer-0.10/libmfw_gst_aiur_demux.so

Version: 3.0.9

License: LGPL

Source module: gst-fsl-plugins

Binary package: Freescle Gstreamer Multimedia Plugins

Origin URL: http://www.freescale.com

3. MP4 video fails to play using GStreamer pipeline.

gst-launch filesrc location=test.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

Logs:

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_element_class_add_pad_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_new_from_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_activate_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_activatepull_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_activatepush_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_chain_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_event_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_element_add_pad: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_setcaps_function: assertion `GST_IS_PAD (pad)' failed

MFW_GST_V4LSINK_PLUGIN 3.0.9 build on Apr 11 2014 10:15:40.

WARNING: erroneous pipeline: could not link filesrc0 to aiurdemux0

4. I have another yocto setup with 'fsl-image-gui'. Running the same MP4 playback pipeline works fine. The difference I can make out is in the library version of 'aiurdemux' . 'fsl-image-gui' is using 3.0.5 and 'fsl-image-fb' is using 3.0.9 of 'aiurdemux'.

root@imx6qsabresd:/usr/lib/gstreamer-0.10# gst-inspect aiurdemux

Factory Details:

Long name: aiur universal demuxer

Class: Codec/Demuxer

Description: demux container file to video, audio, and subtitle

Author(s): Multimedia Team <shmmmw@freescale.com>

Rank: primary + 1 (257)

Plugin Details:

Name: aiur.imx

Description: aiur universal demux/mux

Filename: /usr/lib/gstreamer-0.10/libmfw_gst_aiur_demux.so

Version: 3.0.5

License: LGPL

Source module: gst-fsl-plugins

Binary package: Freescle Gstreamer Multimedia Plugins

Origin URL: http://www.freescale.com.

Questions:

1. Why most of the compiled plugins are present in 'sysroot' folder instead of '/usr/lib' folder ? Do you think that copying those plugins from 'sysroot' to rootfs may create any issue ? 'aiurdemux' is one thing that I see creating issue right now(after sysroot copy) and other pipelines e.g. RTSP and .264 file decoding works without any problem.

2. Has anyone tested 3.0.9 version of 'aiurdemux' on sabresd board with 'fsl-image-fb' recipe ?

3. How should I fix this issue ?

daiane_angolini · ‎04-23-2014

1. Why most of the compiled plugins are present in 'sysroot' folder instead of '/usr/lib' folder ? Do you think that copying those plugins from 'sysroot' to rootfs may create any issue ? 'aiurdemux' is one thing that I see creating issue right now(after sysroot copy) and other pipelines e.g. RTSP and .264 file decoding works without any problem.

One thing is to build, another is to "install". The folder sysroot is used by BitBake to store a copy of sysroot *during* the build. It's a temp dir and there is no warranty is it's going to be good or not. I mean, the complete directory, all binaries are OK.

2. Has anyone tested 3.0.9 version of 'aiurdemux' on sabresd board with 'fsl-image-fb' recipe ?

Yes I have. In fact I just realized several tests (however with only 3 different files/codecs) using gplay (that uses aiurdemux by default) and it worked perfectly on imx6qsabresd board, however I tested using the FSL Community BSP image called fsl-image-machine-test, for all 3 backends (X11, DFB, WL, X11). And I used master. It was part of my test for next YP release 1.7 And the MM version on master today is 3.0.10.

https://github.com/Freescale/meta-fsl-demos/blob/master/recipes-fsl/packagegroups/packagegroup-fsl-g...

It's not available on your source code (you're using dora, and the file I sent you is only available on master) but in this file you understand how you can add ALL gstreamer plugins available

3. How should I fix this issue ?

Take a loon in this file:

https://github.com/Freescale/meta-fsl-demos/blob/master/recipes-fsl/packagegroups/packagegroup-fsl-g...

Please, report your progress.

sumanranjan · ‎04-24-2014

Thanks Daiane.

2. Has anyone tested 3.0.9 version of 'aiurdemux' on sabresd board with 'fsl-image-fb' recipe ?

Yes I have. In fact I just realized several tests (however with only 3 different files/codecs) using gplay (that uses aiurdemux by default) and it worked perfectly on imx6qsabresd board, however I tested using the FSL Community BSP image called fsl-image-machine-test, for all 3 backends (X11, DFB, WL, X11). And I used master. It was part of my test for next YP release 1.7 And the MM version on master today is 3.0.10.

I am able to play MP4 video file using 'gplay' which should be using 'aiurdemux' by default. However, as I have mentioned in my post above that using the pipeline it does not work.

Have you tested by running the pipeline instead of 'gplay' ? Do you see anything wrong with my pipeline command ?

I will download and test the image you have shared and let you know the results.

Regarding GStreamer plugin present in sysroot instead of rootfs folder. Below is what I did in order to make GStreamer a part of filesystem.

Edit fsl-image-fb.bb
Add ‘packagegroup-fsl-gstreamer’ into IMAGE_INSTALL option.

$ vi fsl-community-bsp/sources/meta-fsl-bsp-release/imx/meta-fsl-demos/recipes-fsl/images/fsl-image-fb.bb

…..

IMAGE_INSTALL += " \

${SOC_IMAGE_INSTALL} \

cpufrequtils \

nano \

packagegroup-fsl-gstreamer \

packagegroup-fsl-tools-testapps \

packagegroup-fsl-tools-benchmark \

"

……

sumanranjan · ‎04-24-2014

I tested with the filesystem you gave in the link : fsl-image-machine-test-imx6qsabresd-20140421-122.rootfs.tar.bz2

Observation:

1. 'gplay' works fine as usual.

2. GStreamer pipeline fails to decode with following error,

root@imx6qsabresd:/unit_tests# gst-launch filesrc location=akiyo.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

MFW_GST_V4LSINK_PLUGIN 3.0.10 build on Apr 19 2014 22:19:35.

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.10

wrapper: 1.0.45(VPUWRAPPER_ARM_LINUX Build on Apr 19 2014 22:17:04)

vpulib: 5.4.20

firmware: 2.3.10.40778

Pipeline is PREROLLING ...

(gst-launch-0.10:1246): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'GST_IS_MINI_OBJECT (mini_object)' failed

....

It continue to throw this error until the playback is over. No Video output on the screen.

Please provide your inputs.

daiane_angolini · ‎04-25-2014

So, when you included all gstreamer packages on the previous release, you was able to get gplay working?

Have you double checked the codec used in the file akiyo.mp4 ?

This file used to be a "pure mp4" file with no "container", when I used this file it was with no demuxer in the gst command line

Could you, please, double check?

I'm sorry for not seeing that before

sumanranjan · ‎04-30-2014

I am not sure what you mean by "pure mp4" file. MP4 itself is a container to host video & audio (h264+aac). I guess you mean "akiyo.mp4" has only h264 video component and no audio component. Even that being the case, we would require 'aiurdemux' to fetch h264 video out from MP4 container. I tried with many MP4 files with video+audio but all result in the same error as above.

LeonardoSandova · ‎04-30-2014

Suman, please enable logging with --gst-debug=*:2,GST_ELEMENT_FACTORY:3 as extra parameter on either gplay and gst-launch. This will allow us to see any error and the elements being instantiated.

sumanranjan · ‎05-04-2014

I was on vacation so couldn't reply earlier. I run the test on build-fb image and got the below error log.

$ gst-launch --gst-debug=*:2,GST_ELEMENT_FACTORY:3 filesrc location=akiyo.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

0:00:00.084199667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "filesrc"

0:00:00.090680667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "aiurdemux"

(gst-launch-0.10:718): GStreamer-CRITICAL **: gst_debug_log_valist: assertion `category != NULL' failed

0:00:00.095228667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "vpudec"

0:00:00.101644667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "mfw_v4lsink"

0:00:00.112594667 718 0x1145880 WARN mfw_v4lsink mfw_gst_v4lsink.c:303:mfw_gst_v4l2sink_query_support_formats: unrecognized format:[Y444]YUV444

MFW_GST_V4LSINK_PLUGIN 3.0.9 build on Apr 24 2014 17:25:24.

0:00:00.114548667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "pipeline"

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.9

wrapper: 1.0.40(VPUWRAPPER_ARM_LINUX Build on Apr 24 2014 16:53:25)

vpulib: 5.4.18

firmware: 2.3.10.40778

Pipeline is PREROLLING ...

0:00:00.163790000 718 0x117b4f0 WARN GST_SCHEDULING /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstpad.c:5183:gst_pad_pull_range:<aiurdemux0:sink> pullrange returned buffer of different caps