Gstreamer h264 decoding latency on iMX53

[kartiknatarajan]

I am receiving a h.264 stream over a network onto my i.mx53 board, resizing with mfw_isink and displaying it on its LCD. The delay between the (on board CSI ) camera on the transmitter side getting encoded in h.264 by another i.mx53 device & transmitting over wireless and getting displayed on the other device is of the order of 500ms roundabouts.

I transmitted the same stream on a ubuntu x386 machine and played it with a similar gstreamer receive pipeline and displayed on its screen. The delay is about 300 ms or just below it.

What I have tried and has not improved the performance

1) no resizing in mfw_isink

2) try with mfw_v4lsink

3) set bitrate paramter on the transmitter side vpuencoder to 100/200/300/400 kbps

4) set min-latency=true  and framedrop=false in the mfw_vpudecoder

Things complicate further when I plug in a USB camera on the transmitter side and stream out. On the ubuntu machine the delay is 400ms, on device it is close to 900 ms. Reducing the bitrate below 200 kbps worsens the performance. Guess this is the h264 behaviour.

But in all is this a performance issue with the vpudecoder or am I supposed to suspect some other plugin?

Here is the receiver pipeline for reference

gst-launch -v udpsrc caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, profile-level-id=(int)6553622,packetization-mode=(int)1,encoding-name=(string)H264, sprop-parameter-sets=\"Z0IAFKaCxOQA\\,aM44gAA\\=\", payload=96" port=9030 ! .recv_rtp_sink_0 \

    gstrtpbin ! rtph264depay ! mfw_vpudecoder  min-latency=true framedrop=false  ! mfw_isink disp-height=272 disp-width=240 axis-top=0 axis-left=0 sync=false

I have looked into a similar issue and do not understand how to get it here if that is the solution

https://community.freescale.com/message/316538#316538

[jimmychan]

For your information, someone post the command line below is working well. You may try to modify it for H264 to see if it can improve the performance. (https://community.freescale.com/message/274941#274941)

transmitter:

gst-launch-0.10 -v mfw_v4lsrc capture-width=352 capture-height=288  ! queue ! mfw_vpuencoder codec-type=std_mpeg4 ! rtpmp4vpay send-config=true ! queue ! udpsink host=192.168.0.202 port=5002 sync=false async=false

receiver:

gst-launch-0.10 -v udpsrc port=5002 buffer-size=100000 ! queue ! capsfilter caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP4V-ES, profile-level-id=(string)3, config=(string)000001b003000001b59113000001000000012000c888800f50b042414103, payload=(int)96" ! rtpmp4vdepay ! mfw_vpudecoder codec-type=std_mpeg4 ! queue ! mfw_isink sync=false async=false

[kartiknatarajan]

Hi,

This pipeline is identical to the pipelines that I use. The only difference is that I use h264 and the example cited below is for mp4.

My problem is not with pipelines. They are working fine. My issue is with the latency introduced while decoding. Streaming to a PC, the streaming delay is 300ms, but when I stream to mx53 board, the delay bloats to 700ms. Can we solve this latency problem?

Thanks,

N.Kartik

Senior Developer,

Renu Electronics Pvt. Ltd.