We are seeing strange ethernet behavior on the i.MX6.
In our networking setup, we connect a Linux PC and a SabreSD i.MX6 board using a switch (see the following picture)
+----------+
| Ethernet |
|switch |
+----------+
| |
| |
| |
+----------+ +-------------+
| Linux PC | | i.MX6 board |
+----------+ +-------------+
Both network links are 1Gbit full duplex.
The SabreSD board runs mainline Linux 3.10.9 with the PREEMPT_RT patches + FEC driver from 3.14).
On Linux we have started the iperf server (network stress test tool) in UDP mode (command used: “iperf –s –u”).
The Linux PC runs Ubuntu 12. On this PC we:
- ping the i.MX6 board every second
- run the iperf client in UDP mode (command used: “while [ 1 ]; do iperf –c ‘IP address of SabreSD board’ -u -b 100m -t 30 -l 256;sleep 1;done”)
After a while (most of the time a couple of minutes), we see that the i.MX6 board stops replying to the pings from the Linux PC. Closer inspection shows that the FEC in the i.MX6 doesn’t generate receive frame interrupts anymore. The attached screenshots show that the RXF interrupt is enabled. We know that the FEC is still receiving Ethernet frames (because we see the event counters related to frame reception increasing), but no receive frame interrupts are being generated (only MII interrupts).
If you look in the RDAR register in the screenshots you see that it has the value 0 meaning that the FEC cannot write the received frame in main memory because of a lack of available free receive descriptors.
When we ping the Linux PC from the SabreSD board, we see the receive interrupts on the i.MX6 start occurring again. After studying the FEC driver we see that the driver also empties the receive buffer when handling a transmit interrupt. So it seems that the FEC starts generating receive interrupts again when free receive descriptors are available again. Our question now is: how can the FEC get into state where it stops generating receive interrupts and how can we prevent this?
Hi all!
Recently, i get same problem in imx6q, under running in linux v4.1.6, the board cannot receive any packet but can send arp packet to network. After reading replies , i patches with patch from that https://lkml.org/lkml/2016/11/17/945
After that, the problem is gone, but another question is still exist that receiver using udp to receive packet fronm sender with the frequency about 4ms per packet, sometimes receiver has not got any packets from sender in some moment, which happened last one second or two ,even more longer.
And so, after patched the patches of https://lkml.org/lkml/2016/11/17/945,though the G-net workes never be hang, but the value of IEEE_rx_macerr is increasing, so i suspected that the patch is working but not find the root cause of the problem.
any suggestion will be appreciated.
Hello Mustafa, Fabio,
I tried the patches from Russel and unfortunately, I still see the same problems.
The only difference in the test setup is the kernel version. In my first post I mentioned the use of the 3.10.9 kernel with the PREEMPT_RT patches. Now I used the kernel from Russel (3.15.0-rc1+) without PREEMP_RT patches (also because these patches are not available for the 3.15 kernel).
The two debugger screenshots show the FEC registers when the FEC is in the error mode. You see the IEEE_R_FRAME_OK event counter increasing which indicates that frames are received. But the only interrupt being generated is the MII interrupt. I put a breakpoint on the fec_enet_interrupt() function and every time the breakpoint is hit, the FEC ENET_EIR register has the value 0x00800000. You also see the IEEE_R_MACERR event counter increasing which shows that receive FIFO overflows are happening. In my view this is consistent with the RDAR register being 0 (the MAC receives frames but cannot write them to main memory because there are no empty descriptors in the receive ring).
When the FEC has stopped generating receive interrupts, I used the ping command on the Sabre board to send some frames from the Sabre board to the Linux PC. The FEC then starts to generate receive interrupts again. The driver source code shows that the receive buffer is also emptied when a transmit interrupt occurs. So it seems that the FEC stops generating receive interrupts when frames are received and there are no more empty descriptors.
Regards,
Jaccon
