i.MX ProcessorsのトピックRe: Gstreamer aiurdemux parser takes long time with TS RTP Streams

Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Mon, 19 May 2014 18:41:09 GMT

Hi There,

We are using the following pipeline to play (live) TS RTP Streams

gst-launch udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" ! .recv_rtp_sink_0 gstrtpbin latency=10 ! rtpmp2tdepay ! aiurdemux ! vpudec low-latency=true ! mfw_v4lsink

It plays fine, however it takes over 10 seconds before the playback begins. Latency is low, so we are not looking at the system buffering 10 seconds worth of data and then starting the playback. See log below

_______

MFW_GST_V4LSINK_PLUGIN 3.0.11 build on May 12 2014 16:13:01.

Setting pipeline to PAUSED ...

[INFO] Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

plugin: 3.0.11

wrapper: 1.0.46(VPUWRAPPER_ARM_LINUX Build on May 19 2014 10:13:37)

vpulib: 5.4.23

firmware: 3.1.1.46056

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Aiur: 3.0.11

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.19 build on Jan 20 2014 02:03:56

mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

Seekable : No

Size(byte): -1

<--- 10 Second delay here

Mpeg2CreateParser:parser created successfully

Movie Info:

Seekable : No

Live : Yes

Duration : 0:00:00.000000000

ReadMode : File

programs : 1

Track : 2

Track 00 [audio_001034] Enabled

ppid: 1, ppid 52

Duration: 0:00:00.000000000

Language: und

Mime:

audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)7

1625, framed=(boolean)true

Track 01 [video_001031] Enabled

ppid: 1, ppid 49

Duration: 0:00:00.000000000

Language: und

_________________

I’ve narrowed the delay down to this function call in aiurdemux.c

CORE_API (inf, createParser, goto fail, core_ret,

(bool) (demux->clip_info.live), file_cbks, mem_cbks, buf_cbks,

(void *) demux, &handle);

However, it is my understanding that the parser is provided as a binary library which means its not very practical for me to troubleshoot further. I've tried the 4.1.0 LTIB BSP & the 1.0.0 Yocto with the same results.

Any ideas on how to solve this problem ?

Thanks in advance,

/Otto

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

jamesbone — Wed, 28 May 2014 15:24:05 GMT

Hello Otto,

We are discussing your issue internally meanwhile, have you seen this entry:

GStreamer RTP Streaming

Have a great day,
Jaime

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Wed, 28 May 2014 16:30:16 GMT

Thanks for the update Jamie !

Yes, I have seen that page. My pipeline is based on it, minus the audio.

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

karina_valencia — Fri, 30 May 2014 12:56:13 GMT

Re: Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

juangutierrez May 29, 2014 10:32 AM (in response to Juan Antonio Gutierrez Rosas)

I get some stream kind of working, and I was able to see the delay after the below traces in the log

Seekable : No

Size(byte): -1

In my case is around 5 seconds delay. However if I removed the "latency=10" property for the gstrtpbin element, Im not able to see the 5 seconds delay

Would you try removing the latency=10 property on your pipeline, please?

gst-launch udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" ! .recv_rtp_sink_0 gstrtpbin ! rtpmp2tdepay ! aiurdemux ! vpudec low-latency=true ! mfw_v4lsink

Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

juangutierrez — Fri, 30 May 2014 14:57:29 GMT

Can you try another demuxer like mpegtsdemux instead of aiurdemux?

You can get this from the gst-plugins-bad package. You only need to install/copy the libgstmpegdemux.so to your filesystem (No need to install the rest of the libraries, just the mpegtsdemux). I have attached the library for L3.04. Or you can build it by your own too.

sudo cp <gst-plugins-bad_directory>/libgstmpegdemux.so rootfs/usr/lib/gstreamer-0.10/

Restart the board and try to run again with the next pipeline

gst-launch udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" ! .recv_rtp_sink_0 gstrtpbin latency=10 ! rtpmp2tdepay ! mpegtsdemux ! vpudec low-latency=true ! mfw_v4lsink

Anyways, the aiurdemux should be working though.

Do you know if there is anyway to reproduce the problem without need for the streaming? I mean just in the sink side.

Or, would you post the pipeline for the source (to stream a video in TS format)? So I can reproduce the problem here.

Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

LeonardoSandova — Fri, 30 May 2014 15:34:13 GMT

Juan/Otto,

do you see the same delay when using playbin2?

Leo

Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 30 May 2014 18:20:22 GMT

Hi Karina,

Unfortunately removing the latency parameter did not make a difference for me. I'll try to come up with a way to reproduce this with a file or maybe a tcpdump replay, so that we have the same source.

Thanks !

/Otto

Re: Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 30 May 2014 18:40:39 GMT

Juan,

Mpegtsdemux does not play my stream, however tsdemux plays it. But we are seeing mainly two problems with tsdemux - Leaks memory and it does not work well with the following pipeline (Play RTP stream then "tee" it to a encoder and save to a file). Playback stutters and the recording falls behind. However, with aiurdemux it works fine (besides a 35s delay during startup). This application is critical for our use of the i.MX6 encoder/decoder.

gst-launch gstrtpbin name=rtpbin latency=10 udpsrc caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" port=5000 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! aiurdemux streaming-latency=10 ! vpudec low-latency=true ! tee name=t t. ! queue ! vpuenc codec=6 bitrate=2000000 ! queue ! matroskamux name=mux ! filesink location=output_media_file.mkv enable-last-buffer=false t. ! queue ! mfw_v4lsink sync=true

So it appears that aiurdemux is a lot closer to working than the mpegtsdemux & tsdemux. My guess is that the aiurdemux startup delay is caused by some form of timeout, which would be pretty straightforward to troubleshoot if we had the source code. Troubleshooting memory leaks and performance problems with tsdemux sounds like it could be a long trip down a dark road.

Any other ideas ?

Thanks in advance,

/Otto

Re: Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 30 May 2014 18:45:17 GMT

Leo,

I'm not sure how to construct a TS RTP pipeline using playbin2, but I have tested this with the same results.

gst-launch -v rtspsrc location=rtsp://10.0.3.102 ! decodebin ! mfw_v4lsink

Thanks,

/Otto

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 30 May 2014 20:47:08 GMT

Interesting update -

I tried recording the stream into a file using the following pipeline

gst-launch gstrtpbin name=rtpbin latency=200 udpsrc caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" port=5000 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! filesink location=video.ts

Then playback using

gst-launch filesrc location=video.ts ! video/mpegts ! aiurdemux ! queue ! vpudec low-latency=true ! mfw_v4lsink

Playback starts immediately, so the problem is not in the data-stream itself. The only difference in the logs (see below) are that the "Live" and "Seekable" flags are different, so it appears to have something do to with that. Is there something else I could try ?

Thanks and have a great weekend,

/Otto

Log with instant playback

--------------------------------

Aiur: 3.0.7

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.14 build on Jun 25 2013 14:37:53

mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

URI:

file:///root/video.ts

Idx File:

/root/.aiur/.root.video.ts.aidx

Seekable : Yes

Size(byte): 1443652

Mpeg2CreateParser:parser created successfully

xc_v4l2_output mxc_v4l2_output.0: Bypass IC.

[1;34mMovie Info:

Seekable : Yes

Live : No

Duration : 0:00:09.130667000

ReadMode : File

programs : 1

Track : 2

Track 00 [audio_001034] Enabled

ppid: 1, ppid 52

Duration: 0:00:09.130667000

Language: und

Mime:

audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)127875, framed=(boolean)true

Track 01 [video_001031] Enabled

ppid: 1, ppid 49

Duration: 0:00:09.111278000

Language: und

Mime:

video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1088, framerate=(fraction)30/1, codec_data=(buffer)000001674d00299a6280f0088fbc05a808080a000007d20001d4c1d0c01ba00001dcd6577971a18037400003b9acaef2e14000

----------------------------------

Log with 15 second delay

--------------------------------

Aiur: 3.0.7

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.14 build on Jun 25 2013 14:37:53

mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

Seekable : No

Size(byte): -1

Mpeg2CreateParser:parser created successfully

Movie Info:

Seekable : No

Live : Yes

Duration : 0:00:00.000000000

ReadMode : File

programs : 1

Track : 2

Track 00 [audio_001034] Enabled

ppid: 1, ppid 52

Duration: 0:0mxc_v4l2_output mxc_v4l2_output.0: Bypass IC.

0:00.000000000

Language: und

Mime:

audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)127875, framed=(boolean)true

Track 01 [video_001031] Enabled

ppid: 1, ppid 49

Duration: 0:00:00.000000000

Language: und

Mime:

-----------------------

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

juangutierrez — Thu, 05 Jun 2014 23:24:07 GMT

I just added some logs wrapping the CORE_API (inf, createParser...) function but actually no observing the delay there.

Other interesting thing is that if I run the sync first (in the i.mx) and then the source (in the PC) I did not see any delay.

Would you post the pipeline used in the source (PC), please?

Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 06 Jun 2014 23:10:59 GMT

Hi Juan,

The stream is coming from a hardware encoder, so its a little bit trickier to replicate. I attached a sample file generated with tcpdump.

1. Install tcpreplay package

2. Then run the command below to adjust the destination IP and MAC of the recorded packets. (Change IP and MAC as needed to match Sabre board)

tcprewrite --fixcsum --infile=ts_rtp_stream.pcap --outfile=ts_rtp_stream2.pcap --dstipmap=0.0.0.0/0:10.0.3.101/32 --enet-dmac=00:11:22:33:44:55

3. Start gstreamer pipeline

4. Start playback of RTP stream (Adjust interface as needed)

sudo tcpreplay --intf1=eth1 ts_rtp_stream2.pcap

That particular one takes about 30 sec to start on my board. When using tsdemux it plays in less than one second.

Let me know how it goes

Thanks for your help

/Otto

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Wed, 25 Jun 2014 16:53:39 GMT

Hi There,

Any updates on this ? Its still a major concern for us.

Thanks in advance,

/Otto

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

gfine — Fri, 27 Jun 2014 20:42:42 GMT

Hi Otto,

There is a workaround but it will never be released into the builds.

The problem stems from the server side. The TS stream's PMT (http://en.wikipedia.org/wiki/MPEG_transport_stream#PMT) tells the receiver's parser that it has a certain number of PIDs (Program IDs). The parser finds one less program than it was told (for example the count in the PMT was 3 PIDs, it finds the first 2 PIDs). It then scans for the last PID (up to 8 MB). This is what causes the latency, and that is how it is currently programmed.

We have a workaround where instead of going into the scan the parser exits the scan. The end results may be mixed depending on what data is on the last PID. We can sent the file with the workaround with the understanding of what it does, and that FSL will not release it.

The ultimate fix is to make sure the sender (server) sends the right data where the PMT PID count and the number of programs (PIDs) are the correct.

Cheers,

Glen

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 27 Jun 2014 21:02:15 GMT

Hi Glen,

Thanks for the information. That makes sense. We do not always have control of the incoming stream so we would like to try out the work-around, even though it will not be released / supported by Freescale.

Have a great weekend !

/Otto

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Fri, 27 Jun 2014 21:44:01 GMT

Btw, after removing the ghost audio stream from the PMT tables it comes right up !

Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

gfine — Fri, 25 Jul 2014 20:03:22 GMT

Hi Otto,

Attached is the workaround file. As mentioned it will not be included in future releases.

Cheers,

Glen

Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

ottob — Mon, 28 Jul 2014 16:37:07 GMT

Thanks a lot Glen ! Works as advertised

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

venkateshm — Tue, 23 Sep 2014 09:59:03 GMT

Hi Glen,

Even with attached library I am getting delay of 30sec.My pipeline is

gst-launch-0.10 udpsrc port=1028 caps=\"application/x-rtp, media=(string)video\" ! rtpmp2tdepay ! aiurdemux name=demux demux. ! queue max-size-time=0 max-size-buffers=0 max-size-bytes=0 ! vpudec ! mfw_isink sync=false async=false disp-height=617 axis-top=57

Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

gfine — Tue, 23 Sep 2014 16:57:34 GMT

Hi Venkatesh,

What is the OS version? This workaround was supplied for 3.0.35.

As stated the problem stems from the source stream having erroneous information about the number of PIDs in the stream. Otherwise your problem is different.

Can you open a seperate thread on the issue?

Cheers,

Glen