Poor Ethernet performance

[chevereau]

Hi,

On different design based on IMX6Q processors, we have rather poor ethernet performance without us being able to explain it.

This example is with a module Apalis iMX6 Quad (Toradex Apalis iMX6 Quad 2GB IT V1.1B) and an Ixora Carrier Board (v1.0a), but we have the same problem with all our design based on IMX6, different rootfs (yocto, embedian...), different kernel release (3.10.17, 4.1...).

To resume :

Command	Ethernet link 1Gb (Mo/s)	Local loop (Mo/s)
scp	10	14
nt & netcat	37	45 Note : with a Celeron(x86), kernel 2.6.33, we have 112 Mo/s
iperf3	51	290

How to explain these differences ? With a 1G Ethernet link, the theoretical maximum speed is 125 MB/s (We know there is a limit on the Ethernet of the IMX6, ~ 400 Mbits/s) . And we do not have a high CPU load.

On local loop, 290 Mo/s with iperf3, only 45 Mo/s with netcat ! where is the problem ?

We performed 3 tests, with an Ethernet link 1Gb and internal local loop :

• Transfert test with scp
• Transfert test with nc and netcat
• iperf3 test

Transfert test - 500 Mo with scp
With another equipment - Ethernet link 1Gb : time scp /tmp/500M root@192.168.1.2:/dev/null The authenticity of host '192.168.1.2 (192.168.1.2)' can't be established. RSA key fingerprint is SHA256:H7v+MGV7dPRPSa8FqE/7GfvTi4TTCfQaPaWED0g+aC0. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.1.2' (RSA) to the list of known hosts. root@192.168.1.2's password: 500M                                          100%  500MB  11.8MB/s   00:42 real    0m46.451s user    0m31.240s sys     0m9.440s Result : ~10Mo/s with Ethernet link 1Gb
With local loop : time scp /tmp/500M root@127.0.0.1:/dev/null 500M                                          100%  500MB  14.3MB/s   00:34 real    0m35.209s user    0m29.260s sys     0m7.510s Result : ~14Mo/s with local loop

With scp, we have encryption. Another way with nc and netcat.

Transfert test - 500 Mo with nc and netcat
With another equipment - Ethernet link 1Gb : netcat –l –p 1234 192.168.1.1 >/tmp/test time nc 192.168.1.2 1234 </tmp/500M real    0m13.522s user    0m0.070s sys     0m6.030s Result : ~37Mo/s with Ethernet link 1Gb
With local loop : netcat -l -p 1234 127.0.0.1 >/tmp/test time nc 127.0.0.1 1234 </tmp/500M real    0m11.049s user    0m0.110s sys     0m2.670s Result : ~45Mo/s with local loop

Another test, with iperf3.

iperf3 test
With another equipment - Ethernet link 1Gb : iperf3 -s iperf3 -c 192.168.1.2 -M 1500 Connecting to host 192.168.1.2, port 5201 [  4] local 192.168.1.1 port 47717 connected to 192.168.1.2 port 5201 [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd [  4]   0.00-1.00   sec  49.3 MBytes   414 Mbits/sec    0    246 KBytes [  4]   1.00-2.00   sec  49.0 MBytes   411 Mbits/sec    0    246 KBytes [  4]   2.00-3.00   sec  49.0 MBytes   411 Mbits/sec    0    257 KBytes [  4]   3.00-4.00   sec  48.8 MBytes   410 Mbits/sec    0    288 KBytes [  4]   4.00-5.00   sec  48.9 MBytes   410 Mbits/sec    0    288 KBytes [  4]   5.00-6.00   sec  48.8 MBytes   409 Mbits/sec    0    304 KBytes [  4]   6.00-7.00   sec  48.9 MBytes   410 Mbits/sec    0    327 KBytes [  4]   7.00-8.00   sec  49.0 MBytes   411 Mbits/sec    0    327 KBytes [  4]   8.00-9.00   sec  48.9 MBytes   410 Mbits/sec    0    327 KBytes [  4]   9.00-10.00  sec  49.0 MBytes   411 Mbits/sec    0    495 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval           Transfer     Bandwidth       Retr [  4]   0.00-10.00  sec   490 MBytes   411 Mbits/sec    0             sender [  4]   0.00-10.00  sec   489 MBytes   410 Mbits/sec                  receiver iperf Done. Result : ~51Mo/s with Ethernet link 1Gb
With local loop : iperf3 -s iperf3 -c 127.0.0.1 -M 1500 Connecting to host 127.0.0.1, port 5201 [  4] local 127.0.0.1 port 45121 connected to 127.0.0.1 port 5201 [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd [  4]   0.00-1.00   sec   277 MBytes  2.32 Gbits/sec    0    485 KBytes [  4]   1.00-2.00   sec   277 MBytes  2.33 Gbits/sec    0    557 KBytes [  4]   2.00-3.00   sec   278 MBytes  2.33 Gbits/sec    0    557 KBytes [  4]   3.00-4.00   sec   277 MBytes  2.33 Gbits/sec    0    618 KBytes [  4]   4.00-5.00   sec   278 MBytes  2.33 Gbits/sec    0    618 KBytes [  4]   5.00-6.00   sec   277 MBytes  2.32 Gbits/sec    0    618 KBytes [  4]   6.00-7.00   sec   278 MBytes  2.32 Gbits/sec    0    618 KBytes [  4]   7.00-8.00   sec   277 MBytes  2.33 Gbits/sec    0    618 KBytes [  4]   8.00-9.00   sec   276 MBytes  2.32 Gbits/sec    0    618 KBytes [  4]   9.00-10.00  sec   277 MBytes  2.32 Gbits/sec    0    618 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval           Transfer     Bandwidth       Retr [  4]   0.00-10.00  sec  2.71 GBytes  2.32 Gbits/sec    0             sender [  4]   0.00-10.00  sec  2.70 GBytes  2.32 Gbits/sec                  receiver iperf Done. Result : ~290Mo/s with local loop

[rans]

Hello CHEVEREAU PASCAL,

Is this issue resolved ?

Is there any advatage in using the PCIe ethernet adapter ?

Thanks,

ranran

[chevereau]

Hello,

The answer was given by NXP (Victor Linnik).

For this function, only one core is used. And with one core we are at the limit of performance.

Using another component does not change anything (that's what we do to get around the 400 Mbps limit on native interfaces).

Pascal Chevereau

[rans]

Hi,

"Another component" is the PCIe ethernet adapter ?

So, there is no advantage in using it and the performance is still poor with or without the external adapter , Right ?

Is there a way to increase performance, or is it that Ethernet performance is limited to ~50MB/sec ?

Thank you,

ranran

[gabrielgrange]

We use i210 with PCI ethernet interface with iMX6 and as the result, the performance was not good as well.
It is ok with iperf or gstreamer but standard commands (like scp, nt & netcat) use one cpu core and provide poor performance with iMX6 IT (800MHz).

Regards

G.Grange

[b36401]

Please note that in case of scp there is a lot of encryption overhead that slows the transfer.
And only 1 core of 4 is loaded with this overhead at one moment. So the load may be only 25%.

Have a great day,
Victor

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[gabrielgrange]

It is strange to have this problem only with iMX6 architecture. I don't know why for the moment.