topic Re: imx53 latency with h264 and udpsrc in i.MX Processors

imx53 latency with h264 and udpsrc

tomrossano — Tue, 06 May 2014 13:48:11 GMT

I am receiving a UDP h264 stream from a device encoding its screen and decoding it with gstreamer on the iMx53. The following is my command line:

gst-launch-0.10 -v udpsrc port=5002 ! queue ! h264parse ! mfw_vpudecoder codec-type=std_avc parser=true ! mfw_isink sync=false async=false > /dev/null

The video output has a latency of about a second, probably because that is the keyframe rate of the encoder. It has very low latency (200-300ms) if I stream it to a Linux PC instead of the iMx53 board using the following command line:

gst-launch-0.10 -v udpsrc port=5002 ! queue ! h264parse ! ffdec_h264 ! autovideosink

Does anyone have any thoughts on how to reduce the latency? Is there something built into the VPU libraries that may be buffering the data in some way that can be optimized?

Thanks.

Tom

Re: imx53 latency with h264 and udpsrc

Wlodek_D_ — Tue, 06 May 2014 13:52:15 GMT

Hello,

Thank you for your post, however please consider moving it to the right community place (e.g. i.MX Community ) to get it visible for active members.

For details please see general advice Where to post a Discussion?

Thank you for using Freescale Community.

Re: imx53 latency with h264 and udpsrc

igorpadykov — Wed, 07 May 2014 03:49:05 GMT

Hi Tom,

had you checked below ?

https://community.freescale.com/message/385122#385122

Re: imx53 latency with h264 and udpsrc

tomrossano — Wed, 07 May 2014 12:16:06 GMT

Thanks. I did not see that one before. I will try some of out the tips in that thread. Unfortunately the video generator is not an rtp stream. It is straight UDP at a variable frame rate.

Re: imx53 latency with h264 and udpsrc

tomrossano — Tue, 13 May 2014 14:27:22 GMT

There is a variable in 'min_data_in_vpu' in gst-fsl-plugin-2.0.3/src/video/vpu_dec.full/src/mfw_gst_vpu_decoder.c and mfw_gst_vpu_thread.c.

I appears that the variable accumulates up to the max ever in the VPU. Setting it to a low value at start (non-720p) and always setting it in mfw_gst_vpudec_copy_data to be

vpu_dec->min_data_in_vpu = (gst_buffer_size + 511) & ~511;

resolved the latency issues I was having. I haven't detected any negative impacts of this change yet.

Re: imx53 latency with h264 and udpsrc

xswyzao — Fri, 22 May 2015 02:23:06 GMT

hi,tomrossano

could u tell me how to modify the code?

this is my sdp file:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=No Title

c=IN IP4 224.124.0.1/5

t=0 0

a=tool:libavformat 56.30.100

m=video 8000 RTP/AVP 96

b=AS:798

a=rtpmap:96 H264/90000

a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAIKzZQEAN/qEAAAMAAQAAAwAyjxgxlg==,aOvssiw=; profile-level-id=640020

m=audio 8002 RTP/AVP 97

b=AS:64

a=rtpmap:97 MPEG4-GENERIC/44100/2

a=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=121056E500

I use "gst-launch-0.10 playbin2 uri=file://`pwd`/test1.sdp

but the video output is very latency！

WARNING: from element /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l: A lot of buffers are being dropped.

Additional debug info:

/mnt/yocto/bin/fsl-release-bsp/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r3/gstreamer-0.10.36/libs/gst/base/gstbasesink.c(2875): gst_base_sink_is_too_late (): /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l:

There may be a timestamping problem, or this computer is too slow.

Re: imx53 latency with h264 and udpsrc

xswyzao — Mon, 25 May 2015 01:35:54 GMT

I find different framerate frame,as 15、20、24、

but the imx53 received stream framerate is always 30.

I copy the mp4 file to sd card, imx53 could get correct framerate.

0:00:00.971237375 1999 0x10e400 DEBUG mfw_vpudecoder mfw_gst_vpu_decoder.c:4506:mfw_gst_vpudec_setcaps: >>VPU_DEC: set framerate nu 30 de 1