topic Re: ffmpeg and vpu library in i.MX Processors

ffmpeg and vpu library

henrymckeggie — Wed, 14 Aug 2013 17:08:10 GMT

I'm trying to use ffmpeg with the vpu library. I'm having trouble with the connection. I'm using the mxc_vpu_test as an example code base, and trying to have ffmpeg read a video stream from a file and having it decoded by the vpu. I'm very new to ffmpeg, and am having a difficult time finding examples and pointers to doing this.

Re: ffmpeg and vpu library

LeonardoSandova — Mon, 19 Aug 2013 15:38:37 GMT

Porting ffmpeg to call VPU is not easy. Have you considered using GStreamer? This way you do not need any code change, you need just to plug gstreamer elements into a pipeline, and execute these with gst-launch. There are plenty of info on the community, so please do a search on it.

Re: ffmpeg and vpu library

henrymckeggie — Mon, 19 Aug 2013 15:45:40 GMT

Thanks. Where do I find the freescale ffmpeg library?

Re: ffmpeg and vpu library

LeonardoSandova — Mon, 19 Aug 2013 16:13:40 GMT

Hi Henry. There is no freescale ffmpeg library, sorry for the confusion. As I said, in case your project needs some vpu decoding/encoding, the easiest is to use gstreamer. ffmpeg plugins are great elements, but not HW accelerated :smileyhappy:

Leo

Re: ffmpeg and vpu library

Tarek — Mon, 19 Aug 2013 16:35:35 GMT

Use ffmpeg to read blocks of data to a buffer then use appsrc gstreamer element to feed the buffers to gstreamer pipeline.

The pipeline will be something like:

appsrc ! vpudec ! mfw_isink

Re: ffmpeg and vpu library

varsmolta — Mon, 19 Aug 2013 16:45:40 GMT

Tarek is right. I was able to use this link as a guide to get appsrc working in imx6: Ghosh blog: gstreamer appsrc in action

Re: Re: ffmpeg and vpu library

Tarek — Mon, 19 Aug 2013 16:51:56 GMT

Example:

Re: Re: ffmpeg and vpu library

LeonardoSandova — Mon, 19 Aug 2013 17:02:18 GMT

Hi Tarek! You rock! Thanks for sharing code.

Leo

Re: ffmpeg and vpu library

henrymckeggie — Tue, 20 Aug 2013 18:38:37 GMT

Thanks. This is all very helpful. I'm looking at the freescale plugin. I've built it in ltib and now I'm wondering how much work it will be to link it with android libraries. We are playing video on the hdmi output using the framebuffer with the vpu libraries and the mxc-test program. This works very well except for the container problem. We want to play h.264 video only. No audio. and I've been having difficulty getting that from a mp4 file to the vpu library.

Re: ffmpeg and vpu library

Tarek — Wed, 21 Aug 2013 06:33:01 GMT

Forget about mxc-test and just use gstreamer. There are some many examples that Leo wrote on this community forum on how to use gstreamer.

This should handle the container, decoding and HDMI display in a very straight forward way.

eg:

gst-launch filesrc location=file.mp4 typefind=true ! aiurdemux ! vpudec ! mfw_v4lsink

Re: ffmpeg and vpu library

varsmolta — Wed, 21 Aug 2013 13:52:18 GMT

If Henry want to link his application in an android environment, shouldn't he be using OpenMax, instead of gstreamer?

Re: ffmpeg and vpu library

henrymckeggie — Thu, 22 Aug 2013 18:06:31 GMT

Thanks again for your help. I have compiled gstreamer and plugins in ltib. I'm running the ltib rootfs on an mx53 board. I'm having problems with it stopping and then it hangs playing nothing when restarted. Requiring a reboot to play again.

Here's my line: gst-launch filesrc location=video.mp4 typefind=true ! aiurdemux ! mfw_vpudecoder ! mfw_v4lsink

Re: ffmpeg and vpu library

LeonardoSandova — Thu, 22 Aug 2013 18:59:37 GMT

How are you stopping the pipeline? ControlC? ControC should stop the pipeline correctly.

Leo

Re: ffmpeg and vpu library

henrymckeggie — Thu, 22 Aug 2013 20:47:59 GMT

Yes. I'm using Ctl-C to stop, sometime I can start it again, and sometimes I can't, and have to reboot. Also it stops on it's own sometimes and then needs to be rebooted again.

Also I'm not able to use fbdevsink as a sink. I get.

WARNING: erroneous pipeline: could not link mfwgstvpu_dec0 to fbdevsink0

Then is exits.

that command line is:

gst-launch filesrc location=video.mp4 typefind=true ! aiurdemux ! mfw_vpudecoder ! fbdevsink

Re: ffmpeg and vpu library

LeonardoSandova — Thu, 22 Aug 2013 22:15:13 GMT

If you do the playback with playbin2, do you get the same error? If yes, then we are in trouble :smileyhappy: Playbin automatically takes the relevant elements, so it would be interesting to see which extra elements is taking. For the fbdevsink, any reason for using it? when using it, mfw_ipucsc needs to be insterted before fbdevsink.

Leo

Re: ffmpeg and vpu library

henrymckeggie — Thu, 22 Aug 2013 23:45:59 GMT

playbin2 seems to work the same. I did just notice that it seems to stop sending signal to the display. All of a sudden the penguin in the top left corner disappears, the display says it's lost it's signal, and I have to reboot to recover.

Also if I add: mfw_ipucsc

gst-launch filesrc location=video.mp4 typefind=true ! aiurdemux ! mfw_vpudecoder ! mfw_ipucsc | fbdevsink

I get:

fbdevsink: command not found

Re: ffmpeg and vpu library

LeonardoSandova — Fri, 23 Aug 2013 16:16:25 GMT

You mean that playbin2 has the same issue? Use ! instead of | when building pipelines :smileyhappy:

Leo

Re: ffmpeg and vpu library

henrymckeggie — Fri, 23 Aug 2013 16:43:57 GMT

Yes. it now looks like it's an issue with the video, in this case the vga just stopping as though the power was turned off.

Re: ffmpeg and vpu library

henrymckeggie — Tue, 03 Sep 2013 19:58:36 GMT

Hi,

Do you have anything further on this? Also are there examples using the framebuffer?

Re: ffmpeg and vpu library

LeonardoSandova — Tue, 03 Sep 2013 20:23:48 GMT

Any reason why you want to use fbdevsink instead of mfw_v4lsink?

Leo