Gstreamer aiurdemux parser takes long time with TS RTP Streams

[ottob]

Hi There,

We are using the following pipeline to play (live) TS RTP Streams

gst-launch udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" ! .recv_rtp_sink_0 gstrtpbin latency=10 ! rtpmp2tdepay ! aiurdemux ! vpudec low-latency=true ! mfw_v4lsink

It plays fine, however it takes over 10 seconds before the playback begins. Latency is low, so we are not looking at the system buffering 10 seconds worth of data and then starting the playback. See log below

_______

MFW_GST_V4LSINK_PLUGIN 3.0.11 build on May 12 2014 16:13:01.

Setting pipeline to PAUSED ...

[INFO]    Product Info: i.MX6Q/D/S

vpudec versions :smileyhappy:

    plugin: 3.0.11

    wrapper: 1.0.46(VPUWRAPPER_ARM_LINUX Build on May 19 2014 10:13:37)

    vpulib: 5.4.23

    firmware: 3.1.1.46056

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Aiur: 3.0.11

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.19  build on Jan 20 2014 02:03:56

  mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

  file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

    Seekable  : No

    Size(byte): -1

<--- 10 Second delay here

Mpeg2CreateParser:parser created successfully

Movie Info:

    Seekable  : No

    Live      : Yes

    Duration  : 0:00:00.000000000

    ReadMode  : File

    programs    : 1

    Track     : 2

Track 00 [audio_001034] Enabled

    ppid: 1, ppid 52

    Duration: 0:00:00.000000000

    Language: und

    Mime:

          audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)7

          1625, framed=(boolean)true

Track 01 [video_001031] Enabled

    ppid: 1, ppid 49

    Duration: 0:00:00.000000000

    Language: und

_________________

I’ve narrowed the delay down to this function call in aiurdemux.c

CORE_API (inf, createParser, goto fail, core_ret,

(bool) (demux->clip_info.live), file_cbks, mem_cbks, buf_cbks,

(void *) demux, &handle);

However, it is my understanding that the parser is provided as a binary library which means its not very practical for me to troubleshoot further. I've tried the 4.1.0 LTIB BSP & the 1.0.0 Yocto with the same results.

Any ideas on how to solve this problem ?

Thanks in advance,

/Otto

[venkateshm]

Hi Glen,

Even with attached library I am getting delay of 30sec.My pipeline is

gst-launch-0.10 udpsrc port=1028 caps=\"application/x-rtp, media=(string)video\" ! rtpmp2tdepay  ! aiurdemux name=demux demux. ! queue  max-size-time=0 max-size-buffers=0 max-size-bytes=0  ! vpudec  ! mfw_isink sync=false async=false disp-height=617 axis-top=57

[gfine]

Hi Venkatesh,

What is the OS version? This workaround was supplied for 3.0.35.

As stated the problem stems from the source stream having erroneous information about the number of PIDs in the stream.  Otherwise your problem is different. 

Can you open a seperate thread on the issue?

Cheers,

Glen

[venkateshm]

Hi Glen,

I am using IMX6 sdp with Ubuntu 12.04 (kernel 3.0.35)  and gstreamer version is 0.10.36.

[gfine]

Hi Venkatesh,

Please open a seperate community thread for handling this.  Please include your setup and a recreation senario in detail, and any items that may be relevant such as logs, videos, etc.

Cheers,

Glen

[ottob]

Interesting update -

I tried recording the stream into a file using the following pipeline

gst-launch gstrtpbin name=rtpbin latency=200 udpsrc caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" port=5000 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! filesink location=video.ts

Then playback using

gst-launch filesrc location=video.ts ! video/mpegts ! aiurdemux ! queue ! vpudec low-latency=true ! mfw_v4lsink

Playback starts immediately, so the problem is not in the data-stream itself. The only difference in the logs (see below) are that the "Live" and "Seekable" flags are different, so it appears to have something do to with that. Is there something else I could try ?

Thanks and have a great weekend,

/Otto

Log with instant playback

--------------------------------

Aiur: 3.0.7

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.14  build on Jun 25 2013 14:37:53

  mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

  file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

    URI:

          file:///root/video.ts

    Idx File:

          /root/.aiur/.root.video.ts.aidx

    Seekable  : Yes

    Size(byte): 1443652

Mpeg2CreateParser:parser created successfully

xc_v4l2_output mxc_v4l2_output.0: Bypass IC.

[1;34mMovie Info:

    Seekable  : Yes

    Live      : No

    Duration  : 0:00:09.130667000

    ReadMode  : File

    programs    : 1

    Track     : 2

Track 00 [audio_001034] Enabled

    ppid: 1, ppid 52

    Duration: 0:00:09.130667000

    Language: und

    Mime:

          audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)127875, framed=(boolean)true

Track 01 [video_001031] Enabled

    ppid: 1, ppid 49

    Duration: 0:00:09.111278000

    Language: und

    Mime:

          video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1088, framerate=(fraction)30/1, codec_data=(buffer)000001674d00299a6280f0088fbc05a808080a000007d20001d4c1d0c01ba00001dcd6577971a18037400003b9acaef2e14000

---------------------------------- 

Log with 15 second delay

--------------------------------

Aiur: 3.0.7

Core: BLN_MAD-MMLAYER_MPG2PARSER_04.04.14  build on Jun 25 2013 14:37:53

  mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa

  file: /usr/lib/imx-mm/parser/lib_mpg2_parser_arm11_elinux.so.3.1

Content Info:

    Seekable  : No

    Size(byte): -1

Mpeg2CreateParser:parser created successfully

Movie Info:

    Seekable  : No

    Live      : Yes

    Duration  : 0:00:00.000000000

    ReadMode  : File

    programs    : 1

    Track     : 2

Track 00 [audio_001034] Enabled

    ppid: 1, ppid 52

    Duration: 0:0mxc_v4l2_output mxc_v4l2_output.0: Bypass IC.

0:00.000000000

    Language: und

    Mime:

          audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)127875, framed=(boolean)true

Track 01 [video_001031] Enabled

    ppid: 1, ppid 49

    Duration: 0:00:00.000000000

    Language: und

    Mime:

          video/x-h264, parsed=(boolean)true, width=(int)1920, height=(int)1088, framerate=(fraction)30/1, codec_data=(buffer)000001674d00299a6280f0088fbc05a808080a000007d20001d4c1d0c01ba00001dcd6577971a18037400003b9acaef2e14000

-----------------------

[juangutierrez]

I just added some logs wrapping the CORE_API (inf, createParser...) function but actually no observing the delay there.

Other interesting thing is that if I run the sync first (in the i.mx) and then the source (in the PC) I did not see any delay.

Would you post the pipeline used in the source (PC), please?

[ottob]

Hi Juan,

The stream is coming from a hardware encoder, so its a little bit trickier to replicate. I attached a sample file generated with tcpdump.

1. Install tcpreplay package

2. Then run the command below to adjust the destination IP and MAC of the recorded packets. (Change IP and MAC as needed to match Sabre board)

tcprewrite --fixcsum --infile=ts_rtp_stream.pcap --outfile=ts_rtp_stream2.pcap --dstipmap=0.0.0.0/0:10.0.3.101/32 --enet-dmac=00:11:22:33:44:55

3. Start gstreamer pipeline

gst-launch gstrtpbin name=rtpbin latency=200 udpsrc caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" port=5000 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! aiurdemux ! vpudec low-latency=true ! mfw_v4lsink

4. Start playback of RTP stream (Adjust interface as needed)

sudo tcpreplay --intf1=eth1 ts_rtp_stream2.pcap

That particular one takes about 30 sec to start on my board. When using tsdemux it plays in less than one second.

Let me know how it goes

Thanks for your help

/Otto

[ottob]

Hi There,

Any updates on this ? Its still a major concern for us.

Thanks in advance,

/Otto

[gfine · Accepted solution]

Hi Otto,

There is a workaround but it will never be released into the builds.

The problem stems from the server side.  The TS stream's PMT  (http://en.wikipedia.org/wiki/MPEG_transport_stream#PMT) tells the receiver's parser that it has a certain number of PIDs (Program IDs).  The parser finds one less program than it was told (for example the count in the PMT was 3 PIDs, it finds the first 2 PIDs). It then scans for the last PID (up to 8 MB).  This is what causes the latency, and that is how it is currently programmed.

We have a workaround where instead of going into the scan the parser exits the scan. The end results may be mixed depending on what data is on the last PID. We can sent the file with the workaround with the understanding of what it does, and that FSL will not release it.

The ultimate fix is to make sure the sender (server) sends the right data where the PMT PID count and the number of programs (PIDs) are the correct.

Cheers,

Glen

[ottob]

Hi Glen,

Thanks for the information. That makes sense. We do not always have control of the incoming stream so we would like to try out the work-around, even though it will not be released / supported by Freescale.

Have a great weekend !

/Otto

[gfine]

Hi Otto,

Attached is the workaround file.  As mentioned it will not be included in future releases.

Cheers,

Glen

[ottob]

Thanks a lot Glen ! Works as advertised

[ottob]

Btw, after removing the ghost audio stream from the PMT tables it comes right up !

[jamesbone]

Hello Otto,

We are discussing your issue internally meanwhile, have you seen this entry:

GStreamer RTP Streaming

Have a great day,
Jaime

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[ottob]

Thanks for the update Jamie !

Yes, I have seen that page. My pipeline is based on it, minus the audio.

[karina_valencia]

Re: Re: Re: Gstreamer aiurdemux parser takes long time with TS RTP Streams

juangutierrez May 29, 2014 10:32 AM (in response to Juan Antonio Gutierrez Rosas)

I get some stream kind of working, and I was able to see the delay after the below traces in the log

        Seekable  : No

        Size(byte): -1

In my case is around 5 seconds delay. However if I removed the "latency=10" property for the gstrtpbin element, Im not able to see the 5 seconds delay

Would you try removing the latency=10 property on your pipeline, please?

gst-launch udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" ! .recv_rtp_sink_0 gstrtpbin ! rtpmp2tdepay ! aiurdemux ! vpudec low-latency=true ! mfw_v4lsink

[ottob]

Hi Karina,

Unfortunately removing the latency parameter did not make a difference for me. I'll try to come up with a way to reproduce this with a file or maybe a tcpdump replay, so that we have the same source.

Thanks !

/Otto

[juangutierrez]

Can you try another demuxer like mpegtsdemux instead of aiurdemux?

You can get this from the gst-plugins-bad package. You only need to install/copy the libgstmpegdemux.so to your filesystem (No need to install the rest of the libraries, just the mpegtsdemux). I have attached the library for L3.04. Or you can build it by your own too.

sudo cp <gst-plugins-bad_directory>/libgstmpegdemux.so  rootfs/usr/lib/gstreamer-0.10/

Restart the board and try to run again with the next pipeline

gst-launch udpsrc port=5000 caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" ! .recv_rtp_sink_0 gstrtpbin latency=10 ! rtpmp2tdepay ! mpegtsdemux ! vpudec low-latency=true ! mfw_v4lsink

Anyways, the aiurdemux should be working though.

Do you know if there is anyway to reproduce the problem without need for the streaming? I mean just in the sink side.

Or, would you post the pipeline for the source (to stream a video in TS format)? So I can reproduce the problem here.

[ottob]

Juan,

Mpegtsdemux does not play my stream, however tsdemux plays it. But we are seeing mainly two problems with tsdemux  - Leaks memory and it does not work well with the following pipeline (Play RTP stream then "tee" it to a encoder and save to a file). Playback stutters and the recording falls behind. However, with aiurdemux it works fine (besides a 35s delay during startup). This application is critical for our use of the i.MX6 encoder/decoder.

gst-launch gstrtpbin name=rtpbin latency=10 udpsrc caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=MP2T-ES, payload=33" port=5000 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! aiurdemux streaming-latency=10 ! vpudec low-latency=true ! tee name=t t. ! queue ! vpuenc codec=6 bitrate=2000000 ! queue ! matroskamux name=mux ! filesink location=output_media_file.mkv enable-last-buffer=false t. ! queue ! mfw_v4lsink sync=true

So it appears that aiurdemux is a lot closer to working than the mpegtsdemux & tsdemux. My guess is that the aiurdemux startup delay is caused by some form of timeout, which would be pretty straightforward to troubleshoot if we had the source code. Troubleshooting memory leaks and performance problems with tsdemux sounds like it could be a long trip down a dark road.

Any other ideas ?

Thanks in advance,

/Otto

[LeonardoSandova]

Juan/Otto,

do you see the same delay when using playbin2?

Leo