i.MX6 Ethernet Overruns

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

i.MX6 Ethernet Overruns

4,302 Views
kal
Contributor II

Has anyone found a solution to the i.MX6 Ethernet overrun problem?

If I run iperf between two i.MX6 boards (200mbps UDP), on the receiving i.MX6 I see up to 500 overruns in one second.

It's pretty simple to reproduce:

* Board 1: "iperf -s"

* Board 2: "iperf -c 10.0.0.2 -r -u -b 200M -t 10"

The receiving side can get hundreds of dropped packets in one second (typically when iperf starts ramping up). This is visible in the ifconfig overrun stats, or you can use Boundary's "devregs" program to view the low level overrun count in the ENET hardware ("sudo devregs ENET_IEEE_R_MACERR").

I've tried both rev 1.0 and 1.2 silicon - the 1.2 silicon looks to be better but still can get, say, 500 overruns in one second. I have also tried using the stock Freescale 4.0.0 kernel and the Boundary 4.0.0 kernel which has some useful patches (the stock kernel passes overruns up to the network stack as truncated packets rather than dropping them).

Even when using a protocol with retransmissions like TCP, when you're getting this level of packet loss the TCP connections can stall for several seconds at a time if fast retransmission fails due to continued overruns.

I've tried fiddling with FIFO levels, raising the NIC priority of the ENET block etc but so far it looks like this problem is insurmountable on i.MX6, and we're perhaps forced to add a PCIe Ethernet adapter in order to get reliable networking. It would be great to hear if people have found ways round this using the internal ENET, however.

Many thanks.

Labels (1)
11 Replies

1,615 Views
jamesbone
NXP TechSupport
NXP TechSupport

Unfortunately this is an Errata of the i.MX6 Enet port and there is no fix at this moment for this case.  You can review the i.MX& Errata Documentation in the following link:

http://cache.freescale.com/files/32bit/doc/errata/IMX6DQCE.pdf?fpsp=1&WT_TYPE=Errata&WT_VENDOR=FREES...

MX6 GbE Errata - explanation


Have a great day,
Jaime

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos

1,615 Views
TomE
Specialist II

How big are the Receive and Transmit FIFOs in the Ethernet Controller in this chip?

"Chapter 23 10/100/1000-Mbps Ethernet MAC (ENET)" is a GENERIC Chapter describing the controller, which can have different implementations when built into different chips. The size of the FIFOs is a "variable" rather than being a "constant".

This is made clear in this section:

i.MX 6Dual/6Quad Applications Processor Reference Manual, Rev. 1, 04/2013

23.4.18 Transmit FIFO Watermark Register (ENET_TFWR)

ENET_TFWR field descriptions

This chip may not support the maximum number of bytes written shown below. See the chip specific

information for the ENET module for this value.

I can't find any "chip specific information" in the 5826-page manual that says how big the FIFOs actually are in this chip.

The above register has 6 bits, counting blocks of 64 BYTES, so it can describe a maximum size of 4032 bytes. Other FIFO-related registers in this chip like "23.4.22 Receive FIFO Section Full Threshold (ENET_RSFL)" have nine bits counting 64-BIT words, meaning 511*8 bytes or 4088 bytes.

So are the FIFOs the architectural maximum of 4096 bytes, or are they smaller?

Tom

0 Kudos

1,615 Views
jamesbone
NXP TechSupport
NXP TechSupport

Hello  Tom,

Sorry about that misunderstanding of the documentation,  yes the i.MX6Quad/Dual have 4096bytes.

0 Kudos

1,615 Views
TomE
Specialist II

> MX6 GbE Errata - explanation

I click on that and I get the following. Is the location of that post restricted to Freescale employees or something?

Unauthorized


Access to this place or content is restricted. If you think this is a mistake, please contact your administrator or the person who directed you here.
Tom
0 Kudos

1,615 Views
clemensgruber
Contributor III

Jon Nettleton solved the riddle: There's a bug in the iperf3 UDP tests.

I repeated the tests with iperf2 and got the following results:

10 Mbit/s: 0% packet loss

50 Mbit/s: 0.045% packet loss

100 Mbit/s: 0.31% packet loss

200 Mbit/s: 0.64% packet loss

That's fine for me :smileyhappy: And looks normal. Can you reproduce those / Do they look normal to you too?

I should have checked different iperf versions. Those bogus iperf3 results definitely scared me a little bit yesterday..

0 Kudos

1,615 Views
fabio_estevam
NXP Employee
NXP Employee

Hi Clemens,

Glad to know you are getting good results with iperf2 :-)

0 Kudos

1,615 Views
clemensgruber
Contributor III

Hi,

we have the same issue with a i.MX6Q (TO 1.5). Even enabling pause frames does not decrease the packet loss. Above about 141 Mbit/s UDP, the packet loss increases very fast until nearly all packets are lost (IEEE_macerr is increasing rapidly)

I thought, the "workaround" with the additional ENET IRQ on GPIO1_6 would resolve this issue.. but that only solved the enormous ping latencies. Those are much better now, but the RX packet loss is still catastrophic.

Does this mean that if you need UDP protocols, the i.MX6 chip is pretty much useless above 100 Mbit and there is no workaround possible? If so, the errata sheet entry is a huge understatement and it's not just a little performance degradation from 1Gbit to 400 Mbit but a huge packet loss problem, starting much lower than that.

Is there anything we can do about this? Or is the only viable solution to dump the i.MX6 ENET block altogether and use either PCIe or USB ENET MACs instead?

jamesboneYuriMuhin_ngigorpadykovFabioEstevam

I also tried this with the currrent Mainline Linux kernel 4.2-rc7: Same results.

Thanks.

By the way: The "MX6 GbE Errata - explanation" link from the previous post is not accessible.

0 Kudos

1,615 Views
TomE
Specialist II

> Is there anything we can do about this? Or is the only viable solution to dump the i.MX6 ENET block altogether and use either PCIe or USB ENET MACs instead?

USB is unlikely to be able to support Gigabit Ethernet data rates.

Why not program the PHY to negotiate down to 100MBits/sec? That would get rid of the packet loss. At least in the i.MX6 and maybe move it back to the switch.

Tom

0 Kudos

1,615 Views
fabio_estevam
NXP Employee
NXP Employee

Hi Clemens,

Can you post the exact reproduction steps on 4.2-rc7 to the netdev + linux-arm-kernel mailing list?

1,615 Views
clemensgruber
Contributor III

Yes, I sent a message to both mailing lists and added you as CC. Then we'll continue the discussion on one of the lists.

RX packet loss on i.MX6Q running 4.2-rc7

0 Kudos

1,615 Views
fabio_estevam
NXP Employee
NXP Employee

Cool! Thanks, Clemens

0 Kudos