imx53 latency with h264 and udpsrc

cancel
Showing results for 
Search instead for 
Did you mean: 

imx53 latency with h264 and udpsrc

Jump to solution
1,245 Views
tomrossano
Contributor II

I am receiving a UDP h264 stream from a device encoding its screen and decoding it with gstreamer on the iMx53.  The following is my command line:


gst-launch-0.10 -v udpsrc port=5002 ! queue ! h264parse ! mfw_vpudecoder codec-type=std_avc parser=true ! mfw_isink sync=false async=false > /dev/null

The video output has a latency of about a second, probably because that is the keyframe rate of the encoder.  It has very low latency (200-300ms) if I stream it to a Linux PC instead of the iMx53 board using the following command line:

gst-launch-0.10 -v udpsrc port=5002 ! queue ! h264parse ! ffdec_h264 ! autovideosink

Does anyone have any thoughts on how to reduce the latency?  Is there something built into the VPU libraries that may be buffering the data in some way that can be optimized?

Thanks.

Tom

0 Kudos
1 Solution
172 Views
tomrossano
Contributor II

There is a variable in 'min_data_in_vpu' in gst-fsl-plugin-2.0.3/src/video/vpu_dec.full/src/mfw_gst_vpu_decoder.c and mfw_gst_vpu_thread.c.

I appears that the variable accumulates up to the max ever in the VPU.  Setting it to a low value at start (non-720p) and always setting it in mfw_gst_vpudec_copy_data to be

vpu_dec->min_data_in_vpu = (gst_buffer_size + 511) & ~511;

resolved the latency issues I was having.  I haven't detected any negative impacts of this change yet.

View solution in original post

0 Kudos
6 Replies
173 Views
tomrossano
Contributor II

There is a variable in 'min_data_in_vpu' in gst-fsl-plugin-2.0.3/src/video/vpu_dec.full/src/mfw_gst_vpu_decoder.c and mfw_gst_vpu_thread.c.

I appears that the variable accumulates up to the max ever in the VPU.  Setting it to a low value at start (non-720p) and always setting it in mfw_gst_vpudec_copy_data to be

vpu_dec->min_data_in_vpu = (gst_buffer_size + 511) & ~511;

resolved the latency issues I was having.  I haven't detected any negative impacts of this change yet.

View solution in original post

0 Kudos
172 Views
xswyzao
Contributor II

I find different framerate frame,as 15、20、24、

but the imx53 received stream framerate is always 30.

I copy the mp4 file to sd card, imx53 could get correct framerate.

0:00:00.971237375  1999   0x10e400 DEBUG         mfw_vpudecoder mfw_gst_vpu_decoder.c:4506:mfw_gst_vpudec_setcaps: >>VPU_DEC: set framerate nu 30 de 1

0 Kudos
172 Views
xswyzao
Contributor II

hi,tomrossano

         could u tell me how to modify the code?

this is my sdp file:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=No Title

c=IN IP4 224.124.0.1/5

t=0 0

a=tool:libavformat 56.30.100

m=video 8000 RTP/AVP 96

b=AS:798

a=rtpmap:96 H264/90000

a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QAIKzZQEAN/qEAAAMAAQAAAwAyjxgxlg==,aOvssiw=; profile-level-id=640020

m=audio 8002 RTP/AVP 97

b=AS:64

a=rtpmap:97 MPEG4-GENERIC/44100/2

a=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=121056E500

I use "gst-launch-0.10 playbin2 uri=file://`pwd`/test1.sdp

but the video output is very latency!

WARNING: from element /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l: A lot of buffers are being dropped.

Additional debug info:

/mnt/yocto/bin/fsl-release-bsp/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/gstreamer/0.10.36-r3/gstreamer-0.10.36/libs/gst/base/gstbasesink.c(2875): gst_base_sink_is_too_late (): /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l:

There may be a timestamping problem, or this computer is too slow.

0 Kudos
172 Views
igorpadykov
NXP TechSupport
NXP TechSupport
172 Views
tomrossano
Contributor II

Thanks.  I did not see that one before. I will try some of out the tips in that thread.  Unfortunately the video generator is not an rtp stream.  It is straight UDP at a variable frame rate.

0 Kudos
172 Views
Wlodek_D_
Senior Contributor II

Hello,

Thank you for your post, however please consider moving it to the right community place (e.g. i.MX Community ) to get it visible for active members.

For details please see general advice Where to post a Discussion?

Thank you for using Freescale Community.

0 Kudos