GStreamer with video acceleration

frostybeard · ‎08-18-2011

Hi, I've been trying to get GStreamer with video acceleration working on the i.mx515. I am running Ubuntu Maverick on the Genesi Efika.

I see that indeed some GStreamer plugins exist for i.mx:

http://repository.timesys.com/buildsources/g/gst-fsl-plugin/

I have installed various -dev versions of libraries and have gotten pretty far into the build, but my compilation always stops in the parser section, seemingly due to missing header files. I get

aiurregistry.h:59: error: expected specifier-qualifier-list before 'FslParserGetMetaData"

I can't find where FslParserGetMetaData would be defined anywhere, although I did find a header called fsl_parser.h from a tarball called libfsl-mm-core-dev-1.7.2-2.sendai2.tar.gz from an Ubuntu archive. It did not have the definition and it seemed to be old.

Does anyone know where I can find the headers for being able to compile the GStreamer plugin for i.mx515. (Perhaps it is the same steps needed for compiling for the mx53, since the source package requires a PLATFORM= value for the ./config script, which includes MX51, MX53 and so on.

KanHU · ‎11-02-2011

if you use deb proivded by freescale. I think the mfw_v4lsink is ok for use with totem.

the mfw_v4lsink actually have higher rank than the normal ones. So it should be selected for use as video sink

And also please use playbin2 as totem also use it.

Please use "dpkg -l | grep fsl" to check if there's a Freescale provided gstreamer-plugins-base package also be installed.

Since the playbin2's flags property should add NATIVE_VIDEO to enable Direct render between vpu and v4l.

So if you could attach the log, it will be helpful for me to identify your problem

mwsealey · ‎11-02-2011

Actually we don't ship the FSL ones exactly, we compiled and repackaged them and have added some improvements not present in the stock source code, and we have vetoed the "binary codecs" part since the video playback stuff is completely opensource (using the VPU) and GStreamer on Ubuntu (with the plugin finder) can pick out opensource versions of the codecs you need that we don't provide (we are only licensed to ship the MPEG-LA portfolio currently, which means you have to find audio yourself if it's not Ogg or uncompressed PCM).

If you want GStreamer on the Efika all you need is the October image.

We're having a package repository issue right now so the updates aren't available (and a little too clumsy to put somewhere to download with alternative means) but installing from the latest Installer SD card will get you exactly what you need.

KanHU · ‎11-02-2011

if you use deb proivded by freescale. I think the mfw_v4lsink is ok for use with totem.

the mfw_v4lsink actually have higher rank than the normal ones. So it should be selected for use as video sink

And also please use playbin2 as totem also use it.

Please use "dpkg -l | grep fsl" to check if there's a Freescale provided gstreamer-plugins-base package also be installed.

Since the playbin2's flags property should add NATIVE_VIDEO to enable Direct render between vpu and v4l.

So if you could attach the log, it will be helpful for me to identify your problem

frostybeard · ‎09-08-2011

BTW I am using the Ubuntu Maverick dev version of libgstreamer, together with the Freescale libgstreamer version, which was for some earlier version of Ubuntu. That may be what is breaking my dependencies.

Swamy · ‎09-07-2011

Thanks William for the feedback.

frostybeard · ‎09-07-2011

Hi Swamy, I had to download the i.MX Multimedia Codecs from http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=IMX51_SW

I had to use the "force-all" option installing some of the packages and it broke my dependencies and package management system in Maverick.

The Freescale package, IIRC, is missing the dev version of its libgstreamer, which has some of the header files in it that you need. I found some of the missing header files in the kernel source tree and copied them over to /usr/include

Also with some MPEG4 content I get periodic black screens for about 1/2-1 sec. at a time.With 720p I got a distorted picture, but I suspect that it might be my display, which is DVI, but not full HD.

I would like to try to retrace the steps I took, create debs, and make them available, with a patch to the latest GStreamer so that it does not break deps, but I haven't gotten around to it.

Swamy · ‎09-06-2011

Hi William, I saw your problem on 'FslParserGetMetaData'. Now I am also facing the same issue. If you have found answers please let me know. Thanks

frostybeard · ‎08-25-2011

OK now I have HW accelerated video working in Totem. I needed to patch mfw_v4lsink with a color key value for X.

However, when I try the 720p stereo MP4 version of Big Buck Bunny it starts decoding and playing, but the picture is messed up.

So it definitely looks like 720p is being decoded.

frostybeard · ‎08-24-2011

Hi Matt,

Thanks for the answer.

I am happy once I get the promised 720p video playback working, and it seems like I am almost there. That carrot has been dangling way too close in front of me. :-)

William

frostybeard · ‎08-24-2011

At some point I was able to see a semitransparent video picture overlayed on top of the Ubuntu desktop using playbin which used the mfw_v4lsink. The video picture was smooth. It is an XviD video that didn't work previously. So HW video decoding is working.

I tried running Totem. The command line shows the same as playbin. Video is there and the VPU is in use. However, no video picture is visible in Totem. In the terminal I see it is using the V4LSINK (mfw_v4lsink).

I will try to figure out how to get the GStreamer playbin to use ximagesink, since that seems to work very well.

mwsealey · ‎08-23-2011

Daiane Angolini said:

Great!

When using gstreamer + Ubuntu, I think you should try ximagesink instead of mfw_v4lsink

But, you should try to start Totem from some terminal and track its log. Why do you think it is not using HW acceleration?

Hi William, etc.

Actually we prefer mwf_v4lsink here at Genesi. For under 1280x720 that is (above that, it may not render). Right now we have some strange interactions between xvimagesink and Totem so V4L is the only thing that works. And you would need - ideally - our patched GStreamer sources (since we ship Maverick and FSL's MM core is against Lucid, and we have made some improvements and fixed some bugs) and an updated 2D driver (not ready yet).

The problem is, once we published the binaries to someone we have to release it as per the GPL - we are also liable for patent royalty payments. For something that is just a little too quirky to put on a live system right now, I personally am not willing to actually put anything out there.

Genesi absolutely cannot support you on Freescale's raw codec sources..

daiane_angolini · ‎08-23-2011

Great!

When using gstreamer + Ubuntu, I think you should try ximagesink instead of mfw_v4lsink

But, you should try to start Totem from some terminal and track its log. Why do you think it is not using HW acceleration?

frostybeard · ‎08-23-2011

gst-launch-0.10 playbin uri=file:///home/william/videos/starwars.avi

looks like it is working with full hardware acceleration in the vpu with video being decoded (somewhere), but no picture is visible. I am using an Efika MX with HDMI out.

frostybeard · ‎08-22-2011

OK now I found this PDF: http://www.freescale.com.cn/cstory/ftf/2010/pdf/0951.pdf

When I try to use that pipeline in the PDF: gst-launch-0.10 videotestsrc pattern=1 ! autovideosink

or: videotestsrc pattern=1 ! mfw_v4lsink

It starts prerolling and playing without errors, but there's no picture anywhere to be seen.

frostybeard · ‎08-22-2011

Hi Daiane, I found the FSL debs and installed them. Now when I run gst-inspect-0.10 I can see several mfw_ ... plugins.

The (Totem) Movie Player in Ubuntu, however, still does not seem to use an optimized pipleline in gstreamer. I need to work out what kind of pipeline to use with these plugins. I seem to recall reading some PDF from Freescale a while back about GStreamer.

daiane_angolini · ‎08-22-2011

I think you should work with the "really old Ubuntu archives". You can take there any configuration (or even some doc) that will allow you to install the packages you have.

frostybeard · ‎08-20-2011

I can see various freescale libraries (.so). I ran ldconfig and so forth.

Where is the. deb that is released by FSL?

I've looked high and low for GStreamer support for i.mx515. The only things I have found are those on timesys or then really old Ubuntu archives.

daiane_angolini · ‎08-19-2011

Was the binaries copied to the right place? Can you see *.so files? It can be a missing local variable.

Why aren´t you using the deb package released by FSL?

frostybeard · ‎08-19-2011

The previous file was in the Linux headers directory. I copied it over to /usr/include and it worked.

OK I got it compiled. However, GStreamer does not find the new plugins. gst-inpect-0.10 shows none of the new mfw plugins, even if they seemed to install correctly. (they installed to /usr/local/lib)

I found a test program called gplay. gplay seems to play an mp3 file, but nothing is heard.

It gives errors when trying to watch a DVD rip or 720p Mpeg4 video. Video doesn't seem to work at all.

frostybeard · ‎08-19-2011

I definitely got a lot farther this time, but I get this error:

mfw_gst_v4lsink.h:51: fatal error: linux/mxc_v4l2.h: No such file or directory

Apparently I am missing some video4linux2 stuff. I will start digging for that.