How to improve the quality of video streaming from MX6 VPU through WAN?

RobbieJiang — Thu, 19 Mar 2015 05:26:30 GMT

Hi,

We are trying to stream an H.264 encoded video from MX6 to a target host through WAN.

The target host can be reached and the network performance (such as bandwidth, packet drop rate, etc)

between MX6 and the remote host was measured by 'iperf'.

Following are the iperf measure results.

iperf UDP client:

iperf -u -c <remote_host_ip> -t 30 -i 1 -b 200k -p 5004

iperf UDP server:

iperf -u -s -i 1 -p 5004

------------------------------------------------------------

Server listening on UDP port 5004

Receiving 512 byte datagrams

UDP buffer size: 176 KByte (default)

------------------------------------------------------------

[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams

[ 3] 0.0- 1.0 sec 7.50 KBytes 61.4 Kbits/sec 7.197 ms 0/ 15 (0%)

[ 3] 1.0- 2.0 sec 8.00 KBytes 65.5 Kbits/sec 30.068 ms 0/ 16 (0%)

[ 3] 2.0- 3.0 sec 8.50 KBytes 69.6 Kbits/sec 39.943 ms 0/ 17 (0%)

[ 3] 3.0- 4.0 sec 5.50 KBytes 45.1 Kbits/sec 50.298 ms 0/ 11 (0%)

[ 3] 4.0- 5.0 sec 11.5 KBytes 94.2 Kbits/sec 28.516 ms 2/ 25 (8%)

[ 3] 5.0- 6.0 sec 9.00 KBytes 73.7 Kbits/sec 18.496 ms 0/ 18 (0%)

[ 3] 6.0- 7.0 sec 8.00 KBytes 65.5 Kbits/sec 14.185 ms 0/ 16 (0%)

[ 3] 7.0- 8.0 sec 9.00 KBytes 73.7 Kbits/sec 14.121 ms 0/ 18 (0%)

[ 3] 8.0- 9.0 sec 8.00 KBytes 65.5 Kbits/sec 13.028 ms 0/ 16 (0%)

[ 3] 9.0-10.0 sec 9.00 KBytes 73.7 Kbits/sec 14.275 ms 0/ 18 (0%)

[ 3] 10.0-11.0 sec 8.00 KBytes 65.5 Kbits/sec 15.398 ms 0/ 16 (0%)

[ 3] 11.0-12.0 sec 7.50 KBytes 61.4 Kbits/sec 21.128 ms 0/ 15 (0%)

[ 3] 12.0-13.0 sec 1.50 KBytes 12.3 Kbits/sec 47.014 ms 0/ 3 (0%)

[ 3] 13.0-14.0 sec 5.00 KBytes 41.0 Kbits/sec 70.419 ms 0/ 10 (0%)

[ 3] 14.0-15.0 sec 10.5 KBytes 86.0 Kbits/sec 52.538 ms 19/ 40 (48%)

[ 3] 15.0-16.0 sec 9.00 KBytes 73.7 Kbits/sec 27.084 ms 0/ 18 (0%)

[ 3] 16.0-17.0 sec 8.00 KBytes 65.5 Kbits/sec 18.584 ms 0/ 16 (0%)

[ 3] 17.0-18.0 sec 9.00 KBytes 73.7 Kbits/sec 16.952 ms 0/ 18 (0%)

[ 3] 18.0-19.0 sec 8.00 KBytes 65.5 Kbits/sec 14.732 ms 0/ 16 (0%)

[ 3] 19.0-20.0 sec 9.00 KBytes 73.7 Kbits/sec 16.133 ms 0/ 18 (0%)

[ 3] 20.0-21.0 sec 5.50 KBytes 45.1 Kbits/sec 30.969 ms 0/ 11 (0%)

[ 3] 21.0-22.0 sec 11.5 KBytes 94.2 Kbits/sec 28.372 ms 0/ 23 (0%)

[ 3] 22.0-23.0 sec 8.50 KBytes 69.6 Kbits/sec 26.552 ms 0/ 17 (0%)

[ 3] 23.0-24.0 sec 7.00 KBytes 57.3 Kbits/sec 30.240 ms 0/ 14 (0%)

[ 3] 24.0-25.0 sec 10.0 KBytes 81.9 Kbits/sec 21.584 ms 0/ 20 (0%)

[ 3] 25.0-26.0 sec 8.50 KBytes 69.6 Kbits/sec 18.394 ms 0/ 17 (0%)

[ 3] 26.0-27.0 sec 8.50 KBytes 69.6 Kbits/sec 17.797 ms 0/ 17 (0%)

[ 3] 27.0-28.0 sec 8.50 KBytes 69.6 Kbits/sec 14.014 ms 0/ 17 (0%)

[ 3] 28.0-29.0 sec 8.50 KBytes 69.6 Kbits/sec 15.898 ms 0/ 17 (0%)

[ 3] 29.0-30.0 sec 8.50 KBytes 69.6 Kbits/sec 15.459 ms 0/ 17 (0%)

[ 3] 0.0-30.1 sec 246 KBytes 66.8 Kbits/sec 16.422 ms 21/ 512 (4.1%)

------------------------------------------------------------

From these results, it seems that it is basically safe to send UDP packets at the rate of 200Kbps,

with low packet lost rate (about 4%).

In our test, we run a gstreamer pipeline on MX6 to encode the live video into H.264 and then transfer using RTP/UDP.

And on the remote target host, a gstreamer pipeline is running to receive, decode and play the incoming live video.

Following is the receiving pipeline (on remote PC):

Recver pipeline:

gst-launch -v udpsrc port=5004 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAHqaAtBJkAA\\=\\=\\,aM4wpIAA\", ssrc=(uint)1525174040, payload=(int)96, clock-base=(uint)3621853829, seqnum-base=(uint)62250' ! rtph264depay ! decodebin ! xvimagesink sync=false async=false

Sender pipeline is described as follows with different parameters.

1. With some initial default parameters, the video packet lost rate is very high,

even up to 80%~90% (denoted as Lost_Rate in the following description).

In this case, the decoded video quality is unacceptable.

Here is the sender pipeline on MX6 (320x240@15fps) with high video packet lost rate:

gst-launch -v tvsrc device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=576,framerate=15/1' ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420, width=320,height=240,framerate=15/1'! mfw_deinterlacer ! vpuenc codec=avc gopsize=2 ! video/x-h264,width=320,height=240 ! rtph264pay ! udpsink host=<remote_host_ip> port=5004

Note: this pipeline works perfectly when MX6 device and the target host are in the same LAN.

2. To decrease the packet lost rate, we tried to decrease the size of each UDP packet,

by setting "mtu=512" property of element "rtph264pay", to avaid the potential fragmentation of the UDP packets in WAN.

However, with this setting, the video packet lost rate is still high, almost equal to "Lost_Rate", without any improvement.

3. If set the "bitrate" property of element "vpuenc" to "50000"(that is, 50kbps), the packet lost rate is significantly decreased,

alsmot lower than %5. And of course, the decoded video quality is also dramatically improved.

gst-launch -v tvsrc device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=576,framerate=15/1' ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420, width=320,height=240,framerate=15/1'! mfw_deinterlacer ! vpuenc codec=avc bitrate=50000 gopsize=2 ! video/x-h264,width=320,height=240 ! rtph264pay mtu=512 ! udpsink host=<remote_host_ip> port=5004

However, if we change "bitrate" from "50kbps" to "200kbps", the packet loss rate is significantly increased again.

This is quite different from the aforementioned iperf measure results.

Seems that the encoded video packet lost rate is possibly related to the way how the encoded video data is sent out.

Does the "bitrate" property of the "vpuenc" element mean that the MX6 vpu encoder will continuously send out encoded video data at the rate of "bitrate" ?

And what is the correct unit of the "vpuenc" element's "bitrate" property, in "bps" or "kbps"?

According to the source code of "vpuenc" (gst-fsl-plugins-3.0.7/src/video/vpu/src/vpuenc.c), the unit should be "bps", right?

"And if there is no rate control (by setting "bitrate=0"), the pictures are encoded with a quantization parameter

equal to quantParam in EncParam. "

So for my application context, which one is better, rate control with fixed bitrate or without rate control?

4. The "gopsize" property of "vpuenc" can signifcantly affect the decoded video quality on the receiver side,

as there is inevitable packet lost through WAN. So here we set "gopsize" to 2, which means there is one I frame every two frames.

However, if setting "gopsize=1", each frame is encoded as I frame, the decoded video quality is worse than the case when "gopsize=2".

Why? Is it because higher bitrate is required when gopsize is 1 and every single packet loss will cause the corresponding video frame to be corrupted?

Could you please give me some suggestion on the "RIGHT" setting of properties of the sender/receiver pipelines,

in order to get a better decoded video quality on the remote receiver host through WAN?

(such as how to set the correct "bitrate" and "gopsize" property of "vpuenc", or something else)

Regards,

Robbie

Re: How to improve the quality of video streaming from MX6 VPU through WAN?

Yuri — Tue, 31 Mar 2015 08:43:11 GMT

As I can see, in Your Gstreamer pipeline buffering via "queue" element is not used.
Please try it.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

i.MX Processors中的主题 How to improve the quality of video streaming from MX6 VPU through WAN?

How to improve the quality of video streaming from MX6 VPU through WAN?

Re: How to improve the quality of video streaming from MX6 VPU through WAN?