mfw_isink hwbuf allocator zone?

[joshkurland]

Hello,

I am working on making a list player application using Gstreamer on the imx6 Wandboard-quad.  When I use mfw_isink as my sink element, the pipeline takes approximately 5 seconds to build before it can be displayed.  That is terrible for my application, where I want clip to clip transitions to be as seamless as possible.  This delay seems to be independent of the type of video I am playing; static low-res jpeg images, standard definition video, or 1080p with multiple graphics on top, it all takes 5 seconds before it will start to run.  I noticed that shortly before the video plays, I get this message:

hwbuf allocator zone(3133440) created

can not create threadmfw_gst_isink_setcaps:1489

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

VS0 created. in fmt[NV12] win(0,0-1920,1080:1920x1080) out win(0,0-1920,1080:1920x1080)

set deinterlace mode 0

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

hwbuf allocator zone(614400) created

hwbuf allocator zone(614400) destroied.

What does 'hwbuf allocator zone' mean?  Does this have anything to do with the delay I am encountering?  I tried making two separate gstreamer pipelines so that both video clips will be pre-buffered.  Once the first video ends the other will be set to play, but I still get a long delay and this message about hwbuf allocator zone.

What can I do to reduce/remove this delay?  Again, I am using the imx6 Wandboard-quad built with Yocto kernel 3.0.35-4.0.0.  My VPU information is as follows: 

vpudec versions

  plugin: 3.0.8

  wrapper: 1.0.38(VPUWRAPPER_ARM_LINUX Build on Nov  1 2013 14:15:55)

  vpulib: 5.4.15

  firmware: 2.3.10.40778

Thank you,

Josh Kurland

[joshkurland]

Has anybody else experienced this delay when launching a video or looping one?  Any feedback on this would be really great. 

If any Freescale representative or experienced developer sees this thread, could you tell me if it is possible to reduce or eliminate this delay if I go down to the VPU directly, instead of through Gstreamer?  I would prefer to stick with Gstreamer, but I cannot have that kind of delay showing between videos.

Thanks,

Josh Kurland

[LeonardoSandova]

Hi Josh,

I am not an IPU expert so I can not say much about this delay (maybe the creating of HW buffers raise some poll system calls?).

are you planing to render several videos on the same display? if no, then you can use mfw_v4lsink which does not have this delay.

Leo

[joshkurland]

I am making media list player.  Only one video will be playing at a time, but this video may be looping over and over or it may play a second video after it is done.  The reason I am using mfw_isink is that I also need to include a text overlay and a jpeg/png graphic overlay on top of the video.  I have tested the loading delay time using isink without the other overlays, but it takes the same exact time as it would with the overlays in place.  The only performance hit I see when using the multioverlays is the framerate drops tremendously on 1080i/p videos. 

I have been using mfw_v4lsink until very recently, when I switched over to add multioverlay functionality.  Even when using the v4lsink I noticed a slight delay of around 1 second between load times.  Could you recommend any possible optimizations?  I have tried to make two pipelines in gstreamer.  Both pipelines are being built at the same time, then the first pipeline is set to play.  When the first video reaches its end, the second pipeline is changed from PAUSED to PLAYING.  Yet I still see this delay when the second pipeline takes over.  Do you know if this delay is hardware-based?  Or could it be software-related, either something kernel level or something in my code?

[joshkurland · Accepted solution]

Ah, this can be done!  My approach just needed a little tweaking.  I've managed to get the delay down from 5 seconds to around 1 (next goal: zero delay!).  To cut the delay down, I had to use two pipelines.  Both pipelines had to be created at the same time.  Then the first pipeline's state is set to PLAYING. 

Here is where I went wrong:  In order for the vpudec to decode multiple videos, the state of each pipeline must be set.  By default gstreamer will set the state to READY. Instead, right before I set the first pipe to play, I have to set the second pipeline to PAUSED.  Then I just watch for the eos signal from the first video, flip the states of the pipelines, wait for the end of the second video, and so on. 

Now if anybody would have any suggestions on how to remove the delay entirely, I am all ears...

[Tarek]

Hi Josh,

I had a similar delay problem. I solved it by modifying the gst-fsl-plugins.

In short the VPU doesn't start decoding until there is a certain amount of bits in the buffer. The patch sets this amount to zero.

Here is my patch:

Thanks

[joshkurland]

Hi Tarek,

I applied the patch successfully (thanks for confirming that Leo!) and double checked that it was applied to vpudec.c.  I re-flashed my SD card with the new image and ran a simple video-looping code, but I had the same delay as before.  I also didn't notice a difference when I ran my multi-pipeline approach, although I'm not sure if I would notice anything.

The patch was on gst-fsl-plugin version 3.0.7.  I am currently using version 3.0.9.  Do you know if that difference would conflict with anything in your patch?

Thanks,

Josh

[Tarek]

Hi Josh,

I don't think it's version related it's more stream related. Setting the VPU_DEC_CONF_BUFDELAY to zero improves the latency for my streams. It may or may not help in your case.

[joshkurland]

Oh, okay that makes sense.  I will have to get network streaming working soon, hopefully I will see the improvement there.  Thanks!

[joshkurland]

Thanks Tarek, I will give it a try.  When you set the buffer to zero, what kind of load times are you getting? 

I am using Yocto as my build tool.  Can I apply this patch by adding it in to the gst-fsl-plugin directory in meta-fsl-arm?  I'm sure I also need to add the patch to a bbappend somewhere too.

Thanks,

Josh

[LeonardoSandova]

Correct, that is the way. Great patch.

[LeonardoSandova]

Nice Josh. I got other idea but I am not know if it would work:

Create an app, where you stream your videos, one by one (you can try the gst-rtsp-server, pretty easy to hack). Then, you got other pipeline running (playbin2 uri=rtsp://localhost:XXXX), just listening to that streaming port. I am not sure if this will decrease the 1s delay, it is just another approach worth exploring.

Leo