Building gst-fsl-plugins for i.MX6

[Tarek]

Hi,

I'm trying to compile the gst-gsl-plugins from source but the build process is not compiling all the libraries that it should compile. For example  libmfw_gst_aiur_demux.so and libmfw_gst_beep.so are missing.

I believe there is a configuration parameter that I need to pass to the build system in order to compile the missing libraries.

I've tried --enable-aiur --enable-beep with no success!

I'm using yocto project for this project and I have gst-fsl-plugins version 3.0.1 and Linux L3.0.35_12.08.00

If anyone built this before, please post your configuration parameters or attach the config.log file

Thanks

[fengwei]

gst-fsl-plugins have dependence on libfslcodec and libfslparser. Freescale defines unique interfaces for stream format parser and audio codec. The interfaces are fsl_parser.h and fsl_unia.h.

aiur is the demuxer running upon freescale parser, and beep is the audio decoder running upon freescale codec.

The configuration will check the dependency automatically and decide which plugins will be built except you disable them. So if you install the libfslparser and libfslcodec properly, aiur and demuxer will be able to build.

To install libfslparser, just using standard compile sequence "configure -> make -> make install", the headers will be installed to /usr/include/imx-mm/parser, and the libraries is at /usr/lib/imx-mm/parser.

To install libfslcodec, just using standard compile sequence "configure -> make -> make install", the headers will be installed to /usr/include/imx-mm/audio-codec and /usr/include/imx-mm/video-codec, and the libraries is at /usr/lib/imx-mm/audio-codec and /usr/lib/imx-mm/video-codec.

Aiur plugin libraries dependency paths: "libmfw_gst_aiur_demux.so -> /usr/lib/imx-mm/parser/xx.so.x"

Beep plugin libraries dependency paths: "libmfw_gst_beep.so -> /usr/lib/imx-mm/audio-codec/wrap/xx.so.x -> /usr/lib/imx-mm/audio-codec/xx.so"

/usr/lib/imx-mm/parser/xx.so.x and /usr/lib/imx-mm/audio-codec/wrap/xx.so.x is unique layer libraries, and called by aiur and beep plugins in "dlopen" manner, and /usr/lib/imx-mm/audio-codec/xx.so is audio core codec and called by audio wrap in implicit manner. So, if you don't want to set LD_LIBRARY_PATH, it's better to move /usr/lib/imx-mm/audio-codec/*.so* to /usr/lib after "make install". The behavior of freescale video core codec is similar, which is located at /usr/lib/imx-mm/video-codec.

As mentioned before, aiur and beep will load unique libraries by "dlopen", how do they know where the libraries are? In configuration of gst-fsl-plugins, it will check the dependent header and libraries paths by pkg-config, and record them into config files, after "make install", you could find it at /usr/share/aiur_registry.arm11.cf and  /usr/share/beep_registry.arm12.cf. it might be not so correct in cross compiling environment, so that the file path might be wrong. Please verify it after installation.

Don't hesitate to ask if you have any further questions.

[Tarek]

Thanks Wei that was really helpful.

I have installed the codecs and parser in the correct location now and I can build aiur demux as well as the beep audio decoder never the less I  still have problems with gstreamer.

I noticed that libmfw_vpu.so is not available on my system so I'm not sure if this causes the problem!

I guess this library should be build as part of gst-fsl-plugins, right?

I think the source for it is src/video/vpu and there is no binaries build from this directory. There is also other directories which I would like to build as well. /video/h264_dec ,/video/mpeg2_dec,/video/wmv_dec,/video/mpeg4asp_dec and /audio/ac3_dec but there isn't any obvious configuration parameter.

It will very helpful if you can provide more information on how to build these modules.

The error message I'm getting from gstreamer is:

root@imx6qsabrelite:~# gst-launch playbin2 uri=file:///home/root/big_buck_bunny.mp4

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

Missing element: Quicktime demuxer

WARNING: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: No decoder available for type 'video/quicktime, variant=(string)iso'.

Additional debug info:

gsturidecodebin.c(874): unknown_type_cb (): /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0

ERROR: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20: Your GStreamer installation is missing a plug-in.

Additional debug info:

gstdecodebin2.c(3576): gst_decode_bin_expose (): /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20:

no suitable plugins found

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

Freeing pipeline ...

Best  Regards

[fengwei]

From your log, I think aiurdemux was still not working. try "gst-inspect aiurdemux" to verify its sink capabilities. Correct info should be as following:

Pad Templates:

  SINK template: 'sink'

    Availability: Always

    Capabilities:

      application/ogg

      application/x-annodex

      video/x-matroska

      video/webm

      video/quicktime

      audio/x-m4a

      application/x-3gp

      video/mpeg

            mpegversion: [ 1, 2 ]

      video/mpegts

           systemstream: true

      video/x-cdxa

      video/x-flv

      video/x-msvideo

      video/x-ms-asf

From your log, I think aiurdemux was still not working. try "gst-inspect aiurdemux" to verify its sink capabilities. Correct info should be as following:

Pad Templates:

  SINK template: 'sink'

    Availability: Always

    Capabilities:

      application/ogg

      application/x-annodex

      video/x-matroska

      video/webm

      video/quicktime

      audio/x-m4a

      application/x-3gp

      video/mpeg

            mpegversion: [ 1, 2 ]

      video/mpegts

           systemstream: true

      video/x-cdxa

      video/x-flv

      video/x-msvideo

      video/x-ms-asf

Please check /usr/share/aiur_registry.arm11.cf to find out where's the core parser for mp4. My config is:

...

[MOV/MP4/3GP]

mime= video/quicktime; audio/x-m4a; application/x-3gp

library = /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.1

...

If you install libfslparser into right place, it should be at /usr/lib/imx-mm/parser/lib_mp4_parser_arm11_elinux.so.3.1

About libmfw_vpu and other video decoder plugins, it's dependent on VPU HW on your board. If you have VPU HW and installed libvpuwrap properly (check /usr/include/imx-mm/vpu/vpu_wrapper.h), libmfw_vpu will be built. /video/h264_dec ,/video/mpeg2_dec, /video/mpeg4asp_dec are all SW decoder which libmfw_vpu is also capable of, so they won't be built. ,/video/wmv_dec is also SW decoder but with ability wmv7 and wmv8 that libmfw_vpu can't decode, so it will be built if you have libfslmscodec (video part) properly installed (check file /usr/include/imx-mm/video-codec/wmv789_dec_api.h).

ac3_dec is running on ac3 audio codec, which is in libfslac3codec package. So install libfslac3codec before you building ac3_dec gstreamer plugin.

It's a bit hard for building all those on yocto recipes of your own, please be patient :-).

[Tarek]

Ok I've installed the auir correctly and the vpu headers. Now I can play mpeg2/mpeg4/h264 files Thanks for your help. The output from gst-inspect aiurdemux is the now the same as yours except video/x-ms-asf  Do you know why this is missing?

Also there is no audio coming out of the HDMI TV. I tried both mp3 and mp4 file but there is no Audio. For the mp4 video is playing but no audio.

This is the output from gstreamer which looks fine:

root@imx6qsabrelite:~# gst-launch playbin2 uri=file:///home/root/recit.mp3

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

New clock: GstAudioSinkClock

Got EOS from element "playbin20".

Execution ended after 9078933835 ns.

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

Setting pipeline to NULL ...

Freeing pipeline ..

Also aplay output:

root@imx6qsabrelite:~# aplay -c 1 recit.mp3

Playing raw data 'recit.mp3' : Unsigned 8 bit, Rate 8000 Hz, Mono

I think I'm one step from getting  a complete  Yocto recipe for the i.MX6 which is great

[fengwei]

video/x-ms-asf require libfslmsparser to be installed, that's not in freescale standard parser package libfslparser.

Looks like your "gst-launch playbin2 uri=file:///home/root/recit.mp3" run as expected. Can you check if the sound went to 3.5mm headphone?

On imx6q sabrelite board, there are two sound cards, one for 3.5mm jack and another for HDMI. If you want to let the sound go through HDMI, try the following steps (and results on my board are also listed):

# aplay -l

**** List of PLAYBACK Hardware Devices ****

card 0: sgtl5000audio [sgtl5000-audio], device 0: HiFi sgtl5000-0 []

  Subdevices: 1/1

  Subdevice #0: subdevice #0

card 1: imxhdmisoc [imx-hdmi-soc], device 0: IMX HDMI TX mxc-hdmi-soc-0 []

  Subdevices: 1/1

  Subdevice #0: subdevice #0

Now we can find hdmi card is card 1, then we could try plughw:1 for HDMI testing

# speaker-test -c2 -Dplughw:1

You will hear pink noises through HDMI if your HDMI is working, or your HDMI has issues. (Cable, TV, freescale kernel driver..., I'm not sure)

If speaker-test is successful, you can try your mp3 playing

# gst-launch playbin2 uri=file:///home/root/recit.mp3 audio-sink="alsasink device=plughw:1"

Don't play mp3 by aplay, that won't support mp3, and will take mp3 as raw PCM.

[Tarek]

Thanks Wei, The speaker test is working fine so there is no cable, TV or kernel problems. Here is the log:

root@imx6qsabrelite:~# speaker-test -c2 -Dplughw:1

speaker-test 1.0.25

Playback device is plughw:1

Stream parameters are 48000Hz, S16_LE, 2 channels

Using 16 octaves of pink noise

Rate set to 48000Hz (requested 48000Hz)

Buffer size range from 6144 to 6144

Period size range from 768 to 768

Using max buffer size 6144

Periods = 4

was set period_size = 768

was set buffer_size = 6144

0 - Front Left

1 - Front Right

But I still can't play audio with gstreamer. Here is the output from the command you suggested:

root@imx6qsabrelite:~# gst-launch playbin2 uri=file:///home/root/recit.mp3 audio-sink="alsasink device=plughw:1"

Setting pipeline to PAUSED ...

Pipeline is PREROLLING ...

ERROR: from element /GstPlayBin2:playbin20/GstPlaySink:playsink0: Configured audiosink bin0 is not working.

Additional debug info:

gstplaysink.c(1899): gen_audio_chain (): /GstPlayBin2:playbin20/GstPlaySink:playsink0

ERROR: pipeline doesn't want to preroll.

Setting pipeline to NULL ...

Freeing pipeline ...

[fengwei]

Looks like your gstreamer itself has issues.

Can you try something to see if the error log is same as upstairs? Please give me the results about the following.

# gst-launch audiotestsrc ! alsasink device=plughw:1 (listen to HDMI)

# gst-launch audiotestsrc ! alsasink device=plughw:0 (listen to 3.5mm headphone)

# gst-launch playbin2 uri=file:///home/root/recit.mp3 audio-sink="alsasink device=plughw:0 (listen to 3.5mm headphone)

I need you to provide me some log info.

# export GST_DEBUG=2,alsa:5

if "gst-launch audiotestsrc ! alsasink device=plughw:1" is not working

# gst-launch audiotestsrc ! alsasink device=plughw:1

else

# gst-launch playbin2 uri=file:///home/root/recit.mp3 audio-sink="alsasink device=plughw:1"

[Tarek]

Hi Wei,

My Observation:

Audio from headphone is working fine. Both audiotestsrc and mp3 file.

Audio from HDMI works only once. First time I ran the test I can hear the audio from the TV. Any audio I play afterwards doesn't work. I have to reboot the board to get it working again.

Log files attached:

First time test "Which works fine and I can Hear the audio":

gst-audio-First-time.log

Second run "No Audio is coming out of the HDMI TV":

gst-audio-Second-time.log

[fengwei]

From your log, I can see the gstreamer is working as expected. HDMI on imx6 need some cpu load to work normally, I'm not sure what bsp version did you get, I believe the later ones will handle that better. But we can still have some tests on your version.

Can you please try the following:

# echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

which will make the cpu work at top speed. Then test "gst-launch" or "speaker-test" again for HDMI.

[Tarek]

I've got BSP version 12.08

I don't think it's a performance issue because I can always play the audio first time. Even after changing the scaling_governor the behaviour is still the same. Also the CPU load is no more than 3%.

There must be some buffers or caches that needs to be cleared correctly after the first run but that's not happing.

Is there any /proc or /sys parameters that give's us more visibility on the audio status?

[fengwei]

If you can reproduce it by "speaker-test -Dplughw:1 -c2" (I mean the first time ok and the second time fail), I believe it's something driver specific. Maybe you can raise another question to ask for our bsp guys help. I have already mentioned the problem to the driver owner, and I believe he could help.

[Tarek]

Hi Wei,

Yes I can reproduce the problem with speaker-test.

Thank you very much for all your help. I will raise another question and mark this one as answered.

Thanks again.

Tarek

[egremillion]

Hi,

Was this problem ever re-posted and/or resolved?  I have a similar problem and I can't seem to find any solutions.  Don't want to re-post an old question if I can help it...

Thanks,

Ernie