topic Re: IPU multiple video out + Overlay in i.MX Processors

IPU multiple video out + Overlay

Tarek — Tue, 20 Nov 2012 13:26:01 GMT

Hi,

How do I create multiple instances of the IPU for playing multiple video streams each in a separate window at the same time?

Also which instance will display the Overlay GUI which should cover all video windows?

I'm using:

i.MX6Q - Nitrogen board - BSP 12.08 - Openembedded build system - Poky distribution - Yocto project

Thanks

Re: IPU multiple video out + Overlay

FranciscoCarril — Thu, 29 Nov 2012 22:37:04 GMT

Please take a look to these examples:

GStreamer iMX6 Multi-Overlay

Re: IPU multiple video out + Overlay

Tarek — Fri, 30 Nov 2012 08:14:22 GMT

Hi Rancisco,

I can't use Gstreamer because I need to demux the stream using a modified version of FFMPG. Thus I'm using the VPU API's directly. In my application I have a decoder while loop calling av_read_frame() continuously which gives a compressed Packet "packet.data and packet.size". I pass this buffer to VPU and obtain a decoded Frame YUV. I need to pass the output buffer to the display task. And do the same for multiple streams.

Is there anyway I can integrate Gstreamer into the decoder loop to avoid all the VPU - IPU - GPU complications?

If not, is there a more detailed guidance on how to replicate the Gstreamer implantation?

Re: IPU multiple video out + Overlay

LeonardoSandova — Fri, 30 Nov 2012 16:32:52 GMT

Hi Tarek,

Is there a way you can change any of the ffmpeg demuxer GStreamer elements to get the same result? Perhaps this way will take you longer, but once you have the demuxer element ready, you can plug it in into a standard pipeline and use the mfw_isink (Freescale video sink element), where the latter has internally all the logic you want to implement.

Leo

Re: IPU multiple video out + Overlay

Tarek — Wed, 05 Dec 2012 13:16:59 GMT

Hi Leonardo,

I found an interesting gst-plugin that should do the job for me.

ppsrc — Easy way for applications to inject buffers into a pipeline

I found the source for this plugin as part of gst-plugins-base. I hove compiled and included this into my image.

In my image the libraries are installed correctly as expected:

./usr/lib/libgstapp-0.10.la

./usr/lib/libgstapp-0.10.so.0.25.0

./usr/lib/libgstapp-0.10.so

./usr/lib/libgstapp-0.10.so.0

./usr/lib/gstreamer-0.10/libgstapp.la

But Gstreamer doesn't seam to recognize this plugin

#gst-inspect appsrc

No such element or plugin 'appsrc'

Please can you ask the Gstreamer experts to help with installing this plugin?

Thanks,

Tarek

Re: IPU multiple video out + Overlay

LeonardoSandova — Wed, 05 Dec 2012 17:06:55 GMT

Hi Tarek,

I am glad you found a gstreamer option. I think you have three options so gst can find the compiled plug-in:

1. Move your share objects to /usr/lib/gstreamer-0.10 (sudo cp /usr/lib/libgstapp* /usr/lib/gstreamer-0.10) or

2. Check the ./configure script and indicate where the share objects will be installed (do not remember the exact parameter name) or

3. in your pipeline, add the parameter --gst-plugin-path=/usr/lib

Let me know if it works.

Leo

Re: IPU multiple video out + Overlay

Tarek — Wed, 05 Dec 2012 18:21:18 GMT

Hi Leo,

I found the problem, I have installed the development pkg gst-plugins-base-app-dev but missed the main pkg gst-plugins-base-app. Now I can create appsrc element successfully but I still have problems.

I think the gst VPU plugin is unable to recognize the stream type, although when I used the vpu api's it was fine.

I've got the following pipeline:

/*app-source -> vpu-decoder -> mfw_v4lsink-sink*/

gst_element_link ((GstElement*)ctx->appsrc,ctx->vdecoder );

gst_element_link ( ctx->vdecoder, ctx->vdsink );

I'm pushing packets every time I receive need-data signal but after some packet-pushes I get vpudec critical error.

Do I need to add typefind element to the pipeline?

0:00:00.689993667 1669 0x33865580 DEBUG GST_BUS gstbus.c:767:gst_bus_source_dispatch:<bus1> source 0x33864350 handler returns 1

0:00:00.690191833 1669 0x33865580 DEBUG GST_BUS gstbus.c:500:gst_bus_timed_pop_filtered:<bus1> got message 0x33b02cf8, state-changed from avpacket, type mask is 4294967295

0:00:00.690273167 1669 0x33865580 DEBUG GST_BUS gstbus.c:762:gst_bus_source_dispatch:<bus1> source 0x33864350 calling dispatch with state-changed message from element 'avpacket': GstMessageState, old-state=(GstState)GST_STATE_READY, new-state=(GstState)GST_STATE_PAUSED, pending-state=(GstState)GST_STATE_VOID_PENDING;

0:00:00.690561500 1669 0x33865580 DEBUG GST_BUS gstbus.c:767:gst_bus_source_dispatch:<bus1> source 0x33864350 handler returns 1

vdecode.c(852): push_packet

0:00:00.690957833 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c008

vdecode.c(852): push_packet

0:00:00.691188000 1669 0x33864980 DEBUG appsrc gstappsrc.c:988:gst_app_src_create:<avpacket> we have buffer 0x33b0c008 of size 37905

0:00:00.691412833 1669 0x33864980 DEBUG basesrc gstbasesrc.c:2045:gst_base_src_do_sync:<avpacket> we have no clock

0:00:00.691486000 1669 0x33864980 DEBUG basesrc gstbasesrc.c:2231:gst_base_src_get_range:<avpacket> buffer ok

0:00:00.691587500 1669 0x33864980 DEBUG GST_EVENT gstpad.c:5402:gst_pad_send_event:<video-decoder:sink> have event type newsegment

0:00:00.691670500 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c070

0:00:00.691705167 1669 0x33864980 WARN vpudec vpudec.c:1810:gst_vpudec_sink_event: Unsupport newsegment format 2

0:00:00.691823333 1669 0x33864980 DEBUG GST_PADS gstpad.c:5430:gst_pad_send_event:<video-decoder:sink> sent event, result 0

vdecode.c(852): push_packet

0:00:00.692099166 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c0d8

vdecode.c(852): push_packet

0:00:00.692252666 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c140

vdecode.c(852): push_packet

0:00:00.692390000 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c1a8

vdecode.c(852): push_packet

0:00:00.692523166 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c210

vdecode.c(852): push_packet

0:00:00.692649833 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c278

vdecode.c(852): push_packet

0:00:00.693015166 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c2e0

vdecode.c(852): push_packet

0:00:00.693301666 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c348

vdecode.c(852): push_packet

0:00:00.693585500 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c410

vdecode.c(852): push_packet

0:00:00.693864000 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c478

vdecode.c(852): push_packet

0:00:00.694147166 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c4e0

vdecode.c(852): push_packet

0:00:00.694425833 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c548

vdecode.c(852): push_packet

** (console:1669): CRITICAL **: gst_vpudec_chain: assertion `vpudec->context.handle' failed

0:00:00.695730500 1669 0x33864980 INFO basesrc gstbasesrc.c:2562:gst_base_src_loop:<avpacket> pausing after gst_pad_push() = wrong-state

0:00:00.695841500 1669 0x33864980 DEBUG basesrc gstbasesrc.c:2588:gst_base_src_loop:<avpacket> pausing task, reason wrong-state

0:00:00.695928333 1669 0x33864980 DEBUG GST_PADS gstpad.c:5646:gst_pad_pause_task:<avpacket:src> pause task

0:00:00.696020833 1669 0x33864980 DEBUG task gsttask.c:698:gst_task_set_state:<avpacket:src> Changing task 0x33b0b050 to state 2

0:00:00.724539000 1669 0x33865580 DEBUG appsrc gstappsrc.c:1486:gst_app_src_push_buffer_full:<avpacket> queueing buffer 0x33b0c5b0

vdecode.c(852): push_packet

Re: IPU multiple video out + Overlay

LeonardoSandova — Wed, 05 Dec 2012 18:41:12 GMT

Not sure if this helps but the appsrc element has a 'typefind' property, so you try it before inserting the 'typefind' element. Also, you can use the capsfilter element between the appsrc and the decoder. Both ways specify the decoder sink caps

Leo

Re: IPU multiple video out + Overlay

Tarek — Fri, 07 Dec 2012 12:15:47 GMT

Hi Leo,

The typefind element is the only thing that works in my scenario. I still have more problems but I will create a new thread for that.

Thanks a lot for your help