i.MX ProcessorsのトピックRe: Yocto GStreamer Video Problem [fsl-image-fb]

Yocto GStreamer Video Problem [fsl-image-fb]

sumanranjan — Mon, 14 Apr 2014 10:18:03 GMT

Board : imx6sabresd quadcore board

Recipe : fsl-image-fb

Step 1: get the code

$ mkdir fsl-community-bsp

$ cd fsl-community-bsp

$ repo init -u http://github.com/Freescale/fsl-community-bsp-platform -b dora

$ mv .repo/manifest.xml .repo/manifest.orig #remove community manifest

$ cp <location>/fsl-yocto-release-manifest.xml .repo/manifest.xml #replace with freescale manifest.xml within Yocto Release

$ repo sync #sync the repo

$ cp -r sources/meta-fsl-bsp-release/imx/meta-fsl-arm/conf/machine sources/metafsl-arm/conf # This step copies the new machine configuration files into the meta-fsl-arm so that

# community setup can find new machine files.

Step 2: build it

$ MACHINE=imx6qsabresd source setup-environment build-fb

$ cd ..

$ source fsl-setup-release.sh -b build-fb -e fb

$ bitbake fsl-image-fb

Things build OK and graphics OpenGL/ES tests run fine on imx6sabresd board. However, GStreamer has a problem in this image. First of all, I see that only few GStreamer plugins are copied to filesystem rootfs, but many plugins are present in 'sysroot' folder. Why this is so ??

1. gst-inspect command shows very few plugins.

Total count: 4 plugins, 26 features

When I checked in the "sysroot" of the build folder I found many plugins were present : ../fsl-community-bsp/build-fb/tmp/sysroots/imx6qsabresd/usr/lib/gstreamer-0.10. I copied all plugins from sysroot folder

to rootfs folder ( /usr/lib/).

After this gst-inspect shows many plugins.

Total count: 177 plugins (1 blacklist entry not shown), 594 features

After copying plugins from sysroot folder below tests pass.

a. .264 file is playing successfully.

gst-launch filesrc location=FVDO_Plane_4cif.264 typefind=true ! vpudec ! mfw_v4lsink

b. rtsp camera stream is decoded successfully.

gst-launch rtspsrc location=rtsp://10.20.1.48/axis-media/media.amp typefind=true ! gstrtpjitterbuffer ! rtph264depay queue-delay=0 ! capsfilter caps="video/x-h264,width=640,height=480,framerate=(fraction)30/1" ! vpudec low-latency=true ! queue max-size-buffers=0 max-size-time=0 ! mfw_v4lsink sync=false

2. Before copying plugins from sysroot 'aiurdemux' was not even present, but after copy 'aiurdemux' shows up in 'gst-inspect' command and throws following error.

# gst-inspect aiurdemux

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_element_class_add_pad_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_new_from_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_activate_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_activatepull_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_activatepush_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_chain_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_event_function: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_element_add_pad: assertion `GST_IS_PAD (pad)' failed

(gst-inspect-0.10:666): GStreamer-CRITICAL **: gst_pad_set_setcaps_function: assertion `GST_IS_PAD (pad)' failed

Factory Details:

Long name: aiur universal demuxer

Class: Codec/Demuxer

Description: demux container file to video, audio, and subtitle

Author(s): Multimedia Team <shmmmw@freescale.com>

Rank: primary + 1 (257)

Plugin Details:

Name: aiur.imx

Description: aiur universal demux/mux

Filename: /usr/lib/gstreamer-0.10/libmfw_gst_aiur_demux.so

Version: 3.0.9

License: LGPL

Source module: gst-fsl-plugins

Binary package: Freescle Gstreamer Multimedia Plugins

Origin URL: http://www.freescale.com

3. MP4 video fails to play using GStreamer pipeline.

gst-launch filesrc location=test.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

Logs:

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_element_class_add_pad_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_new_from_template: assertion `GST_IS_PAD_TEMPLATE (templ)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_activate_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_activatepull_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_activatepush_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_chain_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_event_function: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_element_add_pad: assertion `GST_IS_PAD (pad)' failed

(gst-launch-0.10:684): GStreamer-CRITICAL **: gst_pad_set_setcaps_function: assertion `GST_IS_PAD (pad)' failed

MFW_GST_V4LSINK_PLUGIN 3.0.9 build on Apr 11 2014 10:15:40.

WARNING: erroneous pipeline: could not link filesrc0 to aiurdemux0

4. I have another yocto setup with 'fsl-image-gui'. Running the same MP4 playback pipeline works fine. The difference I can make out is in the library version of 'aiurdemux' . 'fsl-image-gui' is using 3.0.5 and 'fsl-image-fb' is using 3.0.9 of 'aiurdemux'.

root@imx6qsabresd:/usr/lib/gstreamer-0.10# gst-inspect aiurdemux

Factory Details:

Long name: aiur universal demuxer

Class: Codec/Demuxer

Description: demux container file to video, audio, and subtitle

Author(s): Multimedia Team <shmmmw@freescale.com>

Rank: primary + 1 (257)

Plugin Details:

Name: aiur.imx

Description: aiur universal demux/mux

Filename: /usr/lib/gstreamer-0.10/libmfw_gst_aiur_demux.so

Version: 3.0.5

License: LGPL

Source module: gst-fsl-plugins

Binary package: Freescle Gstreamer Multimedia Plugins

Origin URL: http://www.freescale.com.

Questions:

1. Why most of the compiled plugins are present in 'sysroot' folder instead of '/usr/lib' folder ? Do you think that copying those plugins from 'sysroot' to rootfs may create any issue ? 'aiurdemux' is one thing that I see creating issue right now(after sysroot copy) and other pipelines e.g. RTSP and .264 file decoding works without any problem.

2. Has anyone tested 3.0.9 version of 'aiurdemux' on sabresd board with 'fsl-image-fb' recipe ?

3. How should I fix this issue ?

Re: Yocto GStreamer Video Problem [fsl-image-fb]

daiane_angolini — Wed, 23 Apr 2014 18:29:04 GMT

One thing is to build, another is to "install". The folder sysroot is used by BitBake to store a copy of sysroot *during* the build. It's a temp dir and there is no warranty is it's going to be good or not. I mean, the complete directory, all binaries are OK.

2. Has anyone tested 3.0.9 version of 'aiurdemux' on sabresd board with 'fsl-image-fb' recipe ?

Yes I have. In fact I just realized several tests (however with only 3 different files/codecs) using gplay (that uses aiurdemux by default) and it worked perfectly on imx6qsabresd board, however I tested using the FSL Community BSP image called fsl-image-machine-test, for all 3 backends (X11, DFB, WL, X11). And I used master. It was part of my test for next YP release 1.7 And the MM version on master today is 3.0.10.

https://github.com/Freescale/meta-fsl-demos/blob/master/recipes-fsl/packagegroups/packagegroup-fsl-gstreamer.bb

It's not available on your source code (you're using dora, and the file I sent you is only available on master) but in this file you understand how you can add ALL gstreamer plugins available

3. How should I fix this issue ?

Take a loon in this file:

https://github.com/Freescale/meta-fsl-demos/blob/master/recipes-fsl/packagegroups/packagegroup-fsl-gstreamer.bb

Please, report your progress.

Re: Yocto GStreamer Video Problem [fsl-image-fb]

daiane_angolini — Wed, 23 Apr 2014 18:34:04 GMT

oh, I forgot to say one thing.

You can, for sure, trick the installation system and copy one binary over the sysroot dir to your sdcard. It's not beaustifull, it's not clean, but it works! It works but in case the binary depends on another binary files. And in case of aiurdemux that's the case! It depends on several other .so files, in a tied directory tree.

Re: Yocto GStreamer Video Problem [fsl-image-fb]

daiane_angolini — Wed, 23 Apr 2014 19:06:15 GMT

if you want to take a look in the image I tested you can see it here:

The direct link is the following, but it should not work forever:

http://ci.ossystems.com.br/public/fsl-community-bsp/master/122

For the permanent link, see FSL Community BSP

Re: Yocto GStreamer Video Problem [fsl-image-fb]

sumanranjan — Thu, 24 Apr 2014 10:00:48 GMT

Thanks Daiane.

2. Has anyone tested 3.0.9 version of 'aiurdemux' on sabresd board with 'fsl-image-fb' recipe ?

I am able to play MP4 video file using 'gplay' which should be using 'aiurdemux' by default. However, as I have mentioned in my post above that using the pipeline it does not work.

Have you tested by running the pipeline instead of 'gplay' ? Do you see anything wrong with my pipeline command ?

I will download and test the image you have shared and let you know the results.

Regarding GStreamer plugin present in sysroot instead of rootfs folder. Below is what I did in order to make GStreamer a part of filesystem.

Edit fsl-image-fb.bb
Add ‘packagegroup-fsl-gstreamer’ into IMAGE_INSTALL option.

$ vi fsl-community-bsp/sources/meta-fsl-bsp-release/imx/meta-fsl-demos/recipes-fsl/images/fsl-image-fb.bb

…..

IMAGE_INSTALL += " \

${SOC_IMAGE_INSTALL} \

cpufrequtils \

nano \

packagegroup-fsl-gstreamer \

packagegroup-fsl-tools-testapps \

packagegroup-fsl-tools-benchmark \

……

Re: Yocto GStreamer Video Problem [fsl-image-fb]

sumanranjan — Fri, 25 Apr 2014 04:46:11 GMT

I tested with the filesystem you gave in the link : fsl-image-machine-test-imx6qsabresd-20140421-122.rootfs.tar.bz2

Observation:

1. 'gplay' works fine as usual.

2. GStreamer pipeline fails to decode with following error,

root@imx6qsabresd:/unit_tests# gst-launch filesrc location=akiyo.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

MFW_GST_V4LSINK_PLUGIN 3.0.10 build on Apr 19 2014 22:19:35.

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.10

wrapper: 1.0.45(VPUWRAPPER_ARM_LINUX Build on Apr 19 2014 22:17:04)

vpulib: 5.4.20

firmware: 2.3.10.40778

Pipeline is PREROLLING ...

(gst-launch-0.10:1246): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'GST_IS_MINI_OBJECT (mini_object)' failed

....

It continue to throw this error until the playback is over. No Video output on the screen.

Please provide your inputs.

Re: Yocto GStreamer Video Problem [fsl-image-fb]

daiane_angolini — Fri, 25 Apr 2014 12:55:32 GMT

So, when you included all gstreamer packages on the previous release, you was able to get gplay working?

Have you double checked the codec used in the file akiyo.mp4 ?

This file used to be a "pure mp4" file with no "container", when I used this file it was with no demuxer in the gst command line

Could you, please, double check?

I'm sorry for not seeing that before

Re: Yocto GStreamer Video Problem [fsl-image-fb]

sumanranjan — Wed, 30 Apr 2014 07:26:06 GMT

I am not sure what you mean by "pure mp4" file. MP4 itself is a container to host video & audio (h264+aac). I guess you mean "akiyo.mp4" has only h264 video component and no audio component. Even that being the case, we would require 'aiurdemux' to fetch h264 video out from MP4 container. I tried with many MP4 files with video+audio but all result in the same error as above.

Re: Yocto GStreamer Video Problem [fsl-image-fb]

daiane_angolini — Wed, 30 Apr 2014 16:42:50 GMT

:smileygrin: yes, everything that you said is right :smileygrin:

I must keep a lot of things in my mind, and sometimes (usually) I mix them up. Usually when talking about version number... Sorry.

I would need to reproduce it by myself, but I would only be able to test it on next Monday. If you could not wait I will need to ask for re-assign. Please, let me know.

Re: Yocto GStreamer Video Problem [fsl-image-fb]

LeonardoSandova — Wed, 30 Apr 2014 16:48:25 GMT

Suman, please enable logging with --gst-debug=*:2,GST_ELEMENT_FACTORY:3 as extra parameter on either gplay and gst-launch. This will allow us to see any error and the elements being instantiated.

Re: Yocto GStreamer Video Problem [fsl-image-fb]

sumanranjan — Mon, 05 May 2014 05:21:36 GMT

I was on vacation so couldn't reply earlier. I run the test on build-fb image and got the below error log.

$ gst-launch --gst-debug=*:2,GST_ELEMENT_FACTORY:3 filesrc location=akiyo.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

0:00:00.084199667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "filesrc"

0:00:00.090680667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "aiurdemux"

(gst-launch-0.10:718): GStreamer-CRITICAL **: gst_debug_log_valist: assertion `category != NULL' failed

0:00:00.095228667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "vpudec"

0:00:00.101644667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "mfw_v4lsink"

0:00:00.112594667 718 0x1145880 WARN mfw_v4lsink mfw_gst_v4lsink.c:303:mfw_gst_v4l2sink_query_support_formats: unrecognized format:[Y444]YUV444

MFW_GST_V4LSINK_PLUGIN 3.0.9 build on Apr 24 2014 17:25:24.

0:00:00.114548667 718 0x1145880 INFO GST_ELEMENT_FACTORY /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstelementfactory.c:376:gst_element_factory_create: creating element "pipeline"

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.9

wrapper: 1.0.40(VPUWRAPPER_ARM_LINUX Build on Apr 24 2014 16:53:25)

vpulib: 5.4.18

firmware: 2.3.10.40778

Pipeline is PREROLLING ...

0:00:00.163790000 718 0x117b4f0 WARN GST_SCHEDULING /home/suman/fsl-community-bsp/build-fb/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r2/gstreamer-0.10.36/gst/gstpad.c:5183:gst_pad_pull_range:<aiurdemux0:sink> pullrange returned buffer of different caps