How to improve the quality of video streaming from MX6 VPU through WAN?

[RobbieJiang]

Hi,

We are trying to stream an H.264 encoded video from MX6 to a target host through WAN.

The target host can be reached and the network performance (such as bandwidth, packet drop rate, etc)

between MX6 and the remote host was measured by 'iperf'.

Following are the iperf measure results.

iperf UDP client:

iperf -u -c <remote_host_ip> -t 30 -i 1 -b 200k -p 5004

iperf UDP server:

iperf -u -s -i 1 -p 5004

------------------------------------------------------------

Server listening on UDP port 5004

Receiving 512 byte datagrams

UDP buffer size:  176 KByte (default)

------------------------------------------------------------

[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams

[  3]  0.0- 1.0 sec  7.50 KBytes  61.4 Kbits/sec   7.197 ms    0/   15 (0%)

[  3]  1.0- 2.0 sec  8.00 KBytes  65.5 Kbits/sec  30.068 ms    0/   16 (0%)

[  3]  2.0- 3.0 sec  8.50 KBytes  69.6 Kbits/sec  39.943 ms    0/   17 (0%)

[  3]  3.0- 4.0 sec  5.50 KBytes  45.1 Kbits/sec  50.298 ms    0/   11 (0%)

[  3]  4.0- 5.0 sec  11.5 KBytes  94.2 Kbits/sec  28.516 ms    2/   25 (8%)

[  3]  5.0- 6.0 sec  9.00 KBytes  73.7 Kbits/sec  18.496 ms    0/   18 (0%)

[  3]  6.0- 7.0 sec  8.00 KBytes  65.5 Kbits/sec  14.185 ms    0/   16 (0%)

[  3]  7.0- 8.0 sec  9.00 KBytes  73.7 Kbits/sec  14.121 ms    0/   18 (0%)

[  3]  8.0- 9.0 sec  8.00 KBytes  65.5 Kbits/sec  13.028 ms    0/   16 (0%)

[  3]  9.0-10.0 sec  9.00 KBytes  73.7 Kbits/sec  14.275 ms    0/   18 (0%)

[  3] 10.0-11.0 sec  8.00 KBytes  65.5 Kbits/sec  15.398 ms    0/   16 (0%)

[  3] 11.0-12.0 sec  7.50 KBytes  61.4 Kbits/sec  21.128 ms    0/   15 (0%)

[  3] 12.0-13.0 sec  1.50 KBytes  12.3 Kbits/sec  47.014 ms    0/    3 (0%)

[  3] 13.0-14.0 sec  5.00 KBytes  41.0 Kbits/sec  70.419 ms    0/   10 (0%)

[  3] 14.0-15.0 sec  10.5 KBytes  86.0 Kbits/sec  52.538 ms   19/   40 (48%)

[  3] 15.0-16.0 sec  9.00 KBytes  73.7 Kbits/sec  27.084 ms    0/   18 (0%)

[  3] 16.0-17.0 sec  8.00 KBytes  65.5 Kbits/sec  18.584 ms    0/   16 (0%)

[  3] 17.0-18.0 sec  9.00 KBytes  73.7 Kbits/sec  16.952 ms    0/   18 (0%)

[  3] 18.0-19.0 sec  8.00 KBytes  65.5 Kbits/sec  14.732 ms    0/   16 (0%)

[  3] 19.0-20.0 sec  9.00 KBytes  73.7 Kbits/sec  16.133 ms    0/   18 (0%)

[  3] 20.0-21.0 sec  5.50 KBytes  45.1 Kbits/sec  30.969 ms    0/   11 (0%)

[  3] 21.0-22.0 sec  11.5 KBytes  94.2 Kbits/sec  28.372 ms    0/   23 (0%)

[  3] 22.0-23.0 sec  8.50 KBytes  69.6 Kbits/sec  26.552 ms    0/   17 (0%)

[  3] 23.0-24.0 sec  7.00 KBytes  57.3 Kbits/sec  30.240 ms    0/   14 (0%)

[  3] 24.0-25.0 sec  10.0 KBytes  81.9 Kbits/sec  21.584 ms    0/   20 (0%)

[  3] 25.0-26.0 sec  8.50 KBytes  69.6 Kbits/sec  18.394 ms    0/   17 (0%)

[  3] 26.0-27.0 sec  8.50 KBytes  69.6 Kbits/sec  17.797 ms    0/   17 (0%)

[  3] 27.0-28.0 sec  8.50 KBytes  69.6 Kbits/sec  14.014 ms    0/   17 (0%)

[  3] 28.0-29.0 sec  8.50 KBytes  69.6 Kbits/sec  15.898 ms    0/   17 (0%)

[  3] 29.0-30.0 sec  8.50 KBytes  69.6 Kbits/sec  15.459 ms    0/   17 (0%)

[  3]  0.0-30.1 sec   246 KBytes  66.8 Kbits/sec  16.422 ms   21/  512 (4.1%)

------------------------------------------------------------

From these results, it seems that it is basically safe to send UDP packets at the rate of 200Kbps,

with low packet lost rate (about 4%).

In our test,  we run a gstreamer pipeline on MX6 to encode the live video into H.264 and then transfer using RTP/UDP.

And on the remote target host, a gstreamer pipeline is running to receive, decode and play the incoming live video.

Following is the receiving pipeline (on remote PC):

Recver pipeline:

gst-launch -v udpsrc port=5004 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)\"Z0JAHqaAtBJkAA\\=\\=\\,aM4wpIAA\", ssrc=(uint)1525174040, payload=(int)96, clock-base=(uint)3621853829, seqnum-base=(uint)62250'  ! rtph264depay ! decodebin ! xvimagesink sync=false async=false

Sender pipeline is described as follows with different parameters.

1. With some initial default parameters, the video packet lost rate is very high,

even up to 80%~90% (denoted as Lost_Rate in the following description).

In this case, the decoded video quality is unacceptable.

Here is the sender pipeline on MX6 (320x240@15fps) with high video packet lost rate:

gst-launch -v tvsrc device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=576,framerate=15/1' ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420, width=320,height=240,framerate=15/1'! mfw_deinterlacer ! vpuenc codec=avc gopsize=2 ! video/x-h264,width=320,height=240 ! rtph264pay ! udpsink host=<remote_host_ip> port=5004

Note: this pipeline works perfectly when MX6 device and  the target  host are in the same LAN.

2. To decrease the packet lost rate, we tried to decrease the size of each UDP packet,

by setting "mtu=512" property of element "rtph264pay", to avaid the potential fragmentation of the UDP packets in WAN.

However, with this setting, the video packet lost rate is still high, almost equal to "Lost_Rate", without any improvement.

gst-launch -v tvsrc device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=576,framerate=15/1' ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420, width=320,height=240,framerate=15/1'! mfw_deinterlacer ! vpuenc codec=avc gopsize=2 ! video/x-h264,width=320,height=240 ! rtph264pay mtu=512 ! udpsink host=<remote_host_ip> port=5004

3. If set the "bitrate" property of element "vpuenc" to "50000"(that is, 50kbps), the packet lost rate is significantly decreased,

alsmot lower than %5. And of course, the decoded video quality is also dramatically improved.

gst-launch -v tvsrc device=/dev/video0 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=720,height=576,framerate=15/1' ! mfw_ipucsc ! 'video/x-raw-yuv,format=(fourcc)I420, width=320,height=240,framerate=15/1'! mfw_deinterlacer ! vpuenc codec=avc bitrate=50000 gopsize=2 ! video/x-h264,width=320,height=240 ! rtph264pay mtu=512 ! udpsink host=<remote_host_ip> port=5004

However, if we change "bitrate" from "50kbps" to "200kbps", the packet loss rate is significantly increased again.

This is quite different from the aforementioned iperf measure results.

Seems that the encoded video packet lost rate is possibly related to the way how the encoded video data is sent out.

Does the "bitrate" property of the "vpuenc" element mean that the MX6 vpu encoder will continuously send out encoded video data at the rate of "bitrate" ?

And what is the correct unit of the "vpuenc" element's "bitrate" property, in "bps" or "kbps"?

According to the source code of "vpuenc" (gst-fsl-plugins-3.0.7/src/video/vpu/src/vpuenc.c), the unit should be "bps", right?

"And if there is no rate control (by setting "bitrate=0"), the pictures are encoded with a quantization parameter

equal to quantParam in EncParam. "

So for my application context, which one is better, rate control with fixed bitrate or without rate control?

4. The "gopsize" property of "vpuenc" can signifcantly affect the decoded video quality on the receiver side,

as there is inevitable packet lost through WAN.  So here we set "gopsize" to 2, which means there is one I frame every two frames.

However, if setting "gopsize=1", each frame is encoded as I frame, the decoded video quality is worse than the case when "gopsize=2".

Why? Is it because higher bitrate is required when gopsize is 1 and every single packet loss will cause the corresponding video frame to be corrupted?

Could you please give me some suggestion on the "RIGHT" setting of properties of the sender/receiver pipelines,

in order to get a better decoded video quality on the remote receiver host through WAN?

(such as how to set the correct "bitrate" and "gopsize" property of "vpuenc", or something else)

Regards,

Robbie

[Yuri]

  As I can see, in Your Gstreamer pipeline buffering via "queue" element is not used.
Please try it.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------