topic Re: GStreamer with video acceleration in i.MX Processors

GStreamer with video acceleration

frostybeard — Thu, 18 Aug 2011 09:47:17 GMT

Hi, I've been trying to get GStreamer with video acceleration working on the i.mx515. I am running Ubuntu Maverick on the Genesi Efika.

I see that indeed some GStreamer plugins exist for i.mx:

http://repository.timesys.com/buildsources/g/gst-fsl-plugin/

I have installed various -dev versions of libraries and have gotten pretty far into the build, but my compilation always stops in the parser section, seemingly due to missing header files. I get

aiurregistry.h:59: error: expected specifier-qualifier-list before 'FslParserGetMetaData"

I can't find where FslParserGetMetaData would be defined anywhere, although I did find a header called fsl_parser.h from a tarball called libfsl-mm-core-dev-1.7.2-2.sendai2.tar.gz from an Ubuntu archive. It did not have the definition and it seemed to be old.

Does anyone know where I can find the headers for being able to compile the GStreamer plugin for i.mx515. (Perhaps it is the same steps needed for compiling for the mx53, since the source package requires a PLATFORM= value for the ./config script, which includes MX51, MX53 and so on.

Re: GStreamer with video acceleration

daiane_angolini — Thu, 18 Aug 2011 17:28:03 GMT

What is the http://repository.timesys.com/buildsources/g/gst-fsl-plugin/ version you´ve been using?

Maybe you should update http://repository.timesys.com/buildsources/f/fsl-codecs-common/ and http://repository.timesys.com/buildsources/f/fsl-mm-codeclib/ too.

Re: GStreamer with video acceleration

frostybeard — Thu, 18 Aug 2011 17:43:18 GMT

Thanks Daiane! I will try this. This makes sense. That's probably where the missing headers are.

Re: GStreamer with video acceleration

daiane_angolini — Thu, 18 Aug 2011 19:28:34 GMT

Please, let us know the result ;)

Re: GStreamer with video acceleration

frostybeard — Fri, 19 Aug 2011 09:45:05 GMT

I definitely got a lot farther this time, but I get this error:

mfw_gst_v4lsink.h:51: fatal error: linux/mxc_v4l2.h: No such file or directory

Apparently I am missing some video4linux2 stuff. I will start digging for that.

Re: GStreamer with video acceleration

frostybeard — Fri, 19 Aug 2011 12:03:30 GMT

The previous file was in the Linux headers directory. I copied it over to /usr/include and it worked.

OK I got it compiled. However, GStreamer does not find the new plugins. gst-inpect-0.10 shows none of the new mfw plugins, even if they seemed to install correctly. (they installed to /usr/local/lib)

I found a test program called gplay. gplay seems to play an mp3 file, but nothing is heard.

It gives errors when trying to watch a DVD rip or 720p Mpeg4 video. Video doesn't seem to work at all.

Re: GStreamer with video acceleration

daiane_angolini — Fri, 19 Aug 2011 15:49:38 GMT

Was the binaries copied to the right place? Can you see *.so files? It can be a missing local variable.

Why aren´t you using the deb package released by FSL?

Re: GStreamer with video acceleration

frostybeard — Sat, 20 Aug 2011 10:23:10 GMT

I can see various freescale libraries (.so). I ran ldconfig and so forth.

Where is the. deb that is released by FSL?

I've looked high and low for GStreamer support for i.mx515. The only things I have found are those on timesys or then really old Ubuntu archives.

Re: GStreamer with video acceleration

daiane_angolini — Mon, 22 Aug 2011 13:37:24 GMT

I think you should work with the "really old Ubuntu archives". You can take there any configuration (or even some doc) that will allow you to install the packages you have.

Re: GStreamer with video acceleration

frostybeard — Tue, 23 Aug 2011 05:56:55 GMT

Hi Daiane, I found the FSL debs and installed them. Now when I run gst-inspect-0.10 I can see several mfw_ ... plugins.

The (Totem) Movie Player in Ubuntu, however, still does not seem to use an optimized pipleline in gstreamer. I need to work out what kind of pipeline to use with these plugins. I seem to recall reading some PDF from Freescale a while back about GStreamer.

Re: GStreamer with video acceleration

frostybeard — Tue, 23 Aug 2011 06:17:35 GMT

OK now I found this PDF: http://www.freescale.com.cn/cstory/ftf/2010/pdf/0951.pdf

When I try to use that pipeline in the PDF: gst-launch-0.10 videotestsrc pattern=1 ! autovideosink

or: videotestsrc pattern=1 ! mfw_v4lsink

It starts prerolling and playing without errors, but there's no picture anywhere to be seen.

Re: GStreamer with video acceleration

frostybeard — Tue, 23 Aug 2011 09:06:47 GMT

gst-launch-0.10 playbin uri=file:///home/william/videos/starwars.avi

looks like it is working with full hardware acceleration in the vpu with video being decoded (somewhere), but no picture is visible. I am using an Efika MX with HDMI out.

Re: GStreamer with video acceleration

daiane_angolini — Tue, 23 Aug 2011 19:03:33 GMT

Great!

When using gstreamer + Ubuntu, I think you should try ximagesink instead of mfw_v4lsink

But, you should try to start Totem from some terminal and track its log. Why do you think it is not using HW acceleration?

Re: GStreamer with video acceleration

mwsealey — Wed, 24 Aug 2011 06:10:28 GMT

Daiane Angolini said:

Great!

When using gstreamer + Ubuntu, I think you should try ximagesink instead of mfw_v4lsink

But, you should try to start Totem from some terminal and track its log. Why do you think it is not using HW acceleration?

Hi William, etc.

Actually we prefer mwf_v4lsink here at Genesi. For under 1280x720 that is (above that, it may not render). Right now we have some strange interactions between xvimagesink and Totem so V4L is the only thing that works. And you would need - ideally - our patched GStreamer sources (since we ship Maverick and FSL's MM core is against Lucid, and we have made some improvements and fixed some bugs) and an updated 2D driver (not ready yet).

The problem is, once we published the binaries to someone we have to release it as per the GPL - we are also liable for patent royalty payments. For something that is just a little too quirky to put on a live system right now, I personally am not willing to actually put anything out there.

Genesi absolutely cannot support you on Freescale's raw codec sources..

Re: GStreamer with video acceleration

frostybeard — Wed, 24 Aug 2011 09:24:43 GMT

At some point I was able to see a semitransparent video picture overlayed on top of the Ubuntu desktop using playbin which used the mfw_v4lsink. The video picture was smooth. It is an XviD video that didn't work previously. So HW video decoding is working.

I tried running Totem. The command line shows the same as playbin. Video is there and the VPU is in use. However, no video picture is visible in Totem. In the terminal I see it is using the V4LSINK (mfw_v4lsink).

I will try to figure out how to get the GStreamer playbin to use ximagesink, since that seems to work very well.

Re: GStreamer with video acceleration

frostybeard — Wed, 24 Aug 2011 11:12:42 GMT

Hi Matt,

Thanks for the answer.

I am happy once I get the promised 720p video playback working, and it seems like I am almost there. That carrot has been dangling way too close in front of me. :-)

William

Re: GStreamer with video acceleration

frostybeard — Thu, 25 Aug 2011 12:50:41 GMT

OK now I have HW accelerated video working in Totem. I needed to patch mfw_v4lsink with a color key value for X.

However, when I try the 720p stereo MP4 version of Big Buck Bunny it starts decoding and playing, but the picture is messed up.

So it definitely looks like 720p is being decoded.

Re: GStreamer with video acceleration

Swamy — Tue, 06 Sep 2011 10:51:08 GMT

Hi William, I saw your problem on 'FslParserGetMetaData'. Now I am also facing the same issue. If you have found answers please let me know. Thanks

Re: GStreamer with video acceleration

frostybeard — Wed, 07 Sep 2011 19:15:33 GMT

Hi Swamy, I had to download the i.MX Multimedia Codecs from http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=IMX51_SW

I had to use the "force-all" option installing some of the packages and it broke my dependencies and package management system in Maverick.

The Freescale package, IIRC, is missing the dev version of its libgstreamer, which has some of the header files in it that you need. I found some of the missing header files in the kernel source tree and copied them over to /usr/include

Also with some MPEG4 content I get periodic black screens for about 1/2-1 sec. at a time.With 720p I got a distorted picture, but I suspect that it might be my display, which is DVI, but not full HD.

I would like to try to retrace the steps I took, create debs, and make them available, with a patch to the latest GStreamer so that it does not break deps, but I haven't gotten around to it.

Re: GStreamer with video acceleration

Swamy — Thu, 08 Sep 2011 05:22:45 GMT

Thanks William for the feedback.