i.MX6Q SabreSD Catastrophic Ethernet Latencies

[MOW]

Dear All

I am running the (unmodified) Freescale Android R13.4.1 Demo-Image on a SabreSD board with i.MX6Q (TO 1.2) connected to a 100 MBit Ethernet and I am experiencing catastrophic and wildly varying ping-latencies (pinging a server in our local ethernet):

root@android:/ # busybox ifconfig

eth0      Link encap:Ethernet  HWaddr 00:04:9F:02:33:C7

          inet addr:172.20.26.244  Bcast:172.20.255.255  Mask:255.255.0.0

          inet6 addr: fe80::204:9fff:fe02:33c7/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:17 errors:0 dropped:0 overruns:0 frame:0

          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:1847 (1.8 KiB)  TX bytes:991 (991.0 B)

lo        Link encap:Local Loopback

          inet addr:127.0.0.1  Mask:255.0.0.0

          inet6 addr: ::1/128 Scope:Host

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:64 errors:0 dropped:0 overruns:0 frame:0

          TX packets:64 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:4608 (4.5 KiB)  TX bytes:4608 (4.5 KiB)

root@android:/ # ping 172.20.8.205

PING 172.20.8.205 (172.20.8.205) 56(84) bytes of data.

64 bytes from 172.20.8.205: icmp_seq=1 ttl=64 time=10.5 ms

64 bytes from 172.20.8.205: icmp_seq=2 ttl=64 time=8.65 ms

64 bytes from 172.20.8.205: icmp_seq=3 ttl=64 time=16.6 ms

64 bytes from 172.20.8.205: icmp_seq=4 ttl=64 time=24.6 ms

64 bytes from 172.20.8.205: icmp_seq=5 ttl=64 time=12.6 ms

64 bytes from 172.20.8.205: icmp_seq=6 ttl=64 time=0.599 ms

64 bytes from 172.20.8.205: icmp_seq=7 ttl=64 time=9.97 ms

64 bytes from 172.20.8.205: icmp_seq=8 ttl=64 time=8.70 ms

64 bytes from 172.20.8.205: icmp_seq=9 ttl=64 time=16.6 ms

64 bytes from 172.20.8.205: icmp_seq=10 ttl=64 time=44.6 ms

64 bytes from 172.20.8.205: icmp_seq=11 ttl=64 time=2.72 ms

64 bytes from 172.20.8.205: icmp_seq=12 ttl=64 time=30.6 ms

64 bytes from 172.20.8.205: icmp_seq=13 ttl=64 time=0.283 ms

^C

--- 172.20.8.205 ping statistics ---

13 packets transmitted, 13 received, 0% packet loss, time 12021ms

rtt min/avg/max/mdev = 0.283/14.425/44.657/12.168 ms

root@android:/ # ping 172.20.8.205

PING 172.20.8.205 (172.20.8.205) 56(84) bytes of data.

64 bytes from 172.20.8.205: icmp_seq=1 ttl=64 time=2.67 ms

64 bytes from 172.20.8.205: icmp_seq=2 ttl=64 time=1.79 ms

64 bytes from 172.20.8.205: icmp_seq=3 ttl=64 time=0.327 ms

64 bytes from 172.20.8.205: icmp_seq=4 ttl=64 time=29.9 ms

64 bytes from 172.20.8.205: icmp_seq=5 ttl=64 time=18.6 ms

64 bytes from 172.20.8.205: icmp_seq=6 ttl=64 time=6.83 ms

64 bytes from 172.20.8.205: icmp_seq=7 ttl=64 time=7.03 ms

64 bytes from 172.20.8.205: icmp_seq=8 ttl=64 time=4.30 ms

64 bytes from 172.20.8.205: icmp_seq=9 ttl=64 time=22.7 ms

64 bytes from 172.20.8.205: icmp_seq=10 ttl=64 time=20.6 ms

64 bytes from 172.20.8.205: icmp_seq=11 ttl=64 time=0.290 ms

64 bytes from 172.20.8.205: icmp_seq=12 ttl=64 time=8.85 ms

64 bytes from 172.20.8.205: icmp_seq=13 ttl=64 time=8.35 ms

64 bytes from 172.20.8.205: icmp_seq=14 ttl=64 time=6.70 ms

64 bytes from 172.20.8.205: icmp_seq=15 ttl=64 time=4.66 ms

64 bytes from 172.20.8.205: icmp_seq=16 ttl=64 time=12.6 ms

64 bytes from 172.20.8.205: icmp_seq=17 ttl=64 time=0.834 ms

64 bytes from 172.20.8.205: icmp_seq=18 ttl=64 time=9.69 ms

64 bytes from 172.20.8.205: icmp_seq=19 ttl=64 time=7.69 ms

^C

--- 172.20.8.205 ping statistics ---

19 packets transmitted, 19 received, 0% packet loss, time 18026ms

rtt min/avg/max/mdev = 0.290/9.199/29.970/8.071 ms

root@android:/ # ping 172.20.8.205

PING 172.20.8.205 (172.20.8.205) 56(84) bytes of data.

64 bytes from 172.20.8.205: icmp_seq=1 ttl=64 time=2.66 ms

64 bytes from 172.20.8.205: icmp_seq=2 ttl=64 time=1.74 ms

64 bytes from 172.20.8.205: icmp_seq=3 ttl=64 time=29.7 ms

64 bytes from 172.20.8.205: icmp_seq=4 ttl=64 time=17.6 ms

64 bytes from 172.20.8.205: icmp_seq=5 ttl=64 time=6.59 ms

64 bytes from 172.20.8.205: icmp_seq=6 ttl=64 time=4.65 ms

64 bytes from 172.20.8.205: icmp_seq=7 ttl=64 time=12.6 ms

64 bytes from 172.20.8.205: icmp_seq=8 ttl=64 time=0.680 ms

64 bytes from 172.20.8.205: icmp_seq=9 ttl=64 time=0.245 ms

64 bytes from 172.20.8.205: icmp_seq=10 ttl=64 time=49.9 ms

The same problem can be seen on our own i.MX6Q-based board both running Android and Linux. It doesn't seem to make any difference, whether I connect the boards to a 100 MBit switch, to a 1 GBit switch or via cross-link cable directly to the PC that I ping -- pinging different PCs/servers also doesn't make a difference. These latencies also show up with other protocols and tools other than ping. Using a Linux-kernel with power-management completely disabled also shows this behavior. Even a back-port of the "fec.c"-mainline driver from public Linux v3.8.2 kernel sources still shows this problem. Performance is so bad, that during ssh- or telnet-sessions we keep losing the connection and TFTP and FTP downloads occasionally fail.

Performing a ping-test running Windows-CE7 on our own i.MX6Q-board I get ping-times <1ms, though -- even on the very same board, that shows the ping-latency problems in Linux/Android...

Running the same tests in the same network environment on i.MX53-based boards running Linux/Android (Freescale's QSB/Loco-Board as well as our own design) show expected ping-latencies of <0.5ms, as well.

So the reason might probably be somewhere in i.MX6x-specific code in the kernel... Anybody else noticed similar problems or has any clue, what the cause of the problem might be?

Regards,

Marc

[AlbertT · Accepted solution]

Hello Marc,

I heard that adding "enable_wait_mode=off" in the boot params should solve this issue. I hope this will work for you too.

Best regards,

Jocelyn Kevorkian

[fangmingan]

Hi, Albert and Marc

   Thank both of you. I met the same problem and found your discussion luckily. I tried the method Albert mentioned. The ping time is less 1ms now, it had been about 100ms!

BRs,

Fangming

[MOW]

Dear Jocelyn

This indeed seems to solve the issue. Thanks a lot! I’ve been searching for the cause of this problem for two weeks already and by now completely run out of ideas. Never thought about this…

Thanks again! ☺

Mit freundlichen Grüßen / Best regards

Marc-Oliver Westerburg

Software Engineering

Garz & Fricke GmbH

Tempowerkring 2, 21079 Hamburg - Germany

Amtsgericht Hamburg HRB 60514

Geschäftsführer: Manfred Garz, Matthias Fricke

Phone: +49 (0) 40 791 899 - 55

Fax: +49 40 / 791 899 - 39

www.garz-fricke.com<http://www.garz-fricke.com/>;

Von: jocelynkevorkian mailto:admin@community.freescale.com

Gesendet: Mittwoch, 13. März 2013 16:19

An: Marc-Oliver Westerburg

Betreff: Re: i.MX Community - i.MX6Q SabreSD Catastrophic Ethernet Latencies

http://community.freescale.com/themes/freescale_theme/images/logo.png<https://community.freescale.com/index.jspa>

i.MX6Q SabreSD Catastrophic Ethernet Latencies

created by jocelynkevorkian<https://community.freescale.com/people/jocelynkevorkian> in i.MX Community - View the full discussion<https://community.freescale.com/message/319425#319425>

[AlbertT]

Great !

Don't forget to mark the subject as solved :smileyhappy:

Best regards,

Jocelyn