imx53 latency with h264 and udpsrc

[tomrossano]

I am receiving a UDP h264 stream from a device encoding its screen and decoding it with gstreamer on the iMx53.  The following is my command line:

gst-launch-0.10 -v udpsrc port=5002 ! queue ! h264parse ! mfw_vpudecoder codec-type=std_avc parser=true ! mfw_isink sync=false async=false > /dev/null

The video output has a latency of about a second, probably because that is the keyframe rate of the encoder.  It has very low latency (200-300ms) if I stream it to a Linux PC instead of the iMx53 board using the following command line:

gst-launch-0.10 -v udpsrc port=5002 ! queue ! h264parse ! ffdec_h264 ! autovideosink

Does anyone have any thoughts on how to reduce the latency?  Is there something built into the VPU libraries that may be buffering the data in some way that can be optimized?

Thanks.

Tom

[tomrossano · Accepted solution]

There is a variable in 'min_data_in_vpu' in gst-fsl-plugin-2.0.3/src/video/vpu_dec.full/src/mfw_gst_vpu_decoder.c and mfw_gst_vpu_thread.c.

I appears that the variable accumulates up to the max ever in the VPU.  Setting it to a low value at start (non-720p) and always setting it in mfw_gst_vpudec_copy_data to be

vpu_dec->min_data_in_vpu = (gst_buffer_size + 511) & ~511;

resolved the latency issues I was having.  I haven't detected any negative impacts of this change yet.

[xswyzao]

I find different framerate frame,as 15、20、24、

but the imx53 received stream framerate is always 30.

I copy the mp4 file to sd card, imx53 could get correct framerate.

0:00:00.971237375  1999   0x10e400 DEBUG         mfw_vpudecoder mfw_gst_vpu_decoder.c:4506:mfw_gst_vpudec_setcaps: >>VPU_DEC: set framerate nu 30 de 1

[xswyzao]

hi,tomrossano

         could u tell me how to modify the code?

this is my sdp file:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=No Title

c=IN IP4 224.124.0.1/5

t=0 0

a=tool:libavformat 56.30.100

m=video 8000 RTP/AVP 96

b=AS:798

a=rtpmap:96 H264/90000

a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAIKzZQEAN/qEAAAMAAQAAAwAyjxgxlg==,aOvssiw=; profile-level-id=640020

m=audio 8002 RTP/AVP 97

b=AS:64

a=rtpmap:97 MPEG4-GENERIC/44100/2

a=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=121056E500

I use "gst-launch-0.10 playbin2 uri=file://`pwd`/test1.sdp

but the video output is very latency！

WARNING: from element /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l: A lot of buffers are being dropped.

Additional debug info:

/mnt/yocto/bin/fsl-release-bsp/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r3/gstreamer-0.10.36/libs/gst/base/gstbasesink.c(2875): gst_base_sink_is_too_late (): /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l:

There may be a timestamping problem, or this computer is too slow.

[igorpadykov]

Hi Tom,

had you checked below ?

https://community.freescale.com/message/385122#385122

[tomrossano]

Thanks.  I did not see that one before. I will try some of out the tips in that thread.  Unfortunately the video generator is not an rtp stream.  It is straight UDP at a variable frame rate.

[Wlodek_D_]

Hello,

Thank you for your post, however please consider moving it to the right community place (e.g. i.MX Community ) to get it visible for active members.

For details please see general advice Where to post a Discussion?

Thank you for using Freescale Community.