Network Gbit Performance on TWR-LS1021A - high CPU utilization

reinhardtartler — Tue, 08 Sep 2015 22:42:50 GMT

Using a TWR-LS1021A board with SDK v1.8 (Linux 3.12.370rt51)

I've configured eth0 following the recommendations from Test Procedure

root@ls1021atwr:~# ethtool -C eth0 rx-frames 22

root@ls1021atwr:~# ethtool -C eth0 tx-frames 22

root@ls1021atwr:~# ethtool -C eth0 rx-usecs 32

root@ls1021atwr:~# ethtool -C eth0 tx-usecs 32

root@ls1021atwr:~# ethtool -K eth0 gro on gso on sg on

root@ls1021atwr:~# netperf -H 10.0.0.20 -l 10 -T 1 -c 100 -C 100 -n2 -t TCP_SENDFILE -v 2 -- -C

TCP SENDFILE TEST from 0.0.0.0 () port 0 AF_INET to 10.0.0.20 () port 0 AF_INET : cpu bind

Recv Send Send Utilization Service Demand

Socket Socket Message Elapsed Send Recv Send Recv

Size Size Size Time Throughput local remote local remote

bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB

87380 16384 16384 10.03 886.20 76.23 18.60 14.094 3.439

Alignment Offset Bytes Bytes Sends Bytes Recvs

Local Remote Local Remote Xfered Per Per

Send Recv Send Recv Send (avg) Recv (avg)

8 8 0 0 1.111e+09 16384.00 67799 23129.53 48026

Maximum

Segment

Size (bytes)

1448

In this setup, the other side is a Core2Duo x86 laptop @2.9 Ghz.

Using strace, I can confirm that netperf is indeed using the sendfile64 syscall:

root@ls1021atwr:~# grep -c sendfile64 netperf.strace

36850

The achieved bandwidth is less than we hoped, and the CPU utilization is unacceptably high. This is already after:

Ensuring that all IRQs go to CPU0 (that seems to be the default, distributing eth0 IRQ as described in http://www.freescale.com/infocenter/topic/QORIQSDK/4085174.html seems to worsen performance)
Pin netperf to CPU1

Why is the CPU utilization still so extremely high, and what can be done to improve this further?

Re: Network Gbit Performance on TWR-LS1021A - high CPU utilization

yipingwang — Fri, 11 Sep 2015 08:23:20 GMT

Hello Reinhard Tartler,

For ls1021atwr CPU Rev 1.0 board, when the frame size is 16384 bytes, the throughput data is about 850Mbps, this test result looks normal.

Would you please let me know your expected performance data?

If you are just in the evaluation stage, I suggest you choose ls1021atwr CPU Rev 2.0 board and "Linux SDK for LS1021A v0.4", you will get much better Network performance and lower CPU utilization.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

LayerscapeのトピックRe: Network Gbit Performance on TWR-LS1021A - high CPU utilization

Network Gbit Performance on TWR-LS1021A - high CPU utilization

Re: Network Gbit Performance on TWR-LS1021A - high CPU utilization