IMX6Q enet RX macerr

[庞磊]

Hi,

I ave a custom imx6 based board running linux 4.1.15 and am seeing an issue where if I leave the unit connected to our office network for a day or two, it eventually stops responding to pings.  In ethtool, when this happens I can see the IEEE_rx_macerr count start increasing, before the issue, the count is at 0.  IEEE_rx_frame_ok is also still increasing at this point.

In ifconfig, the rx packet and rx byte counts stops， altogether But can send, the PC side can catch the arp request sent by the board, also reply normally, but the board can not receive

[root@imx6-rcxw /]# ethtool -S eth0
NIC statistics:
tx_dropped: 0
tx_packets: 23174
tx_broadcast: 117
tx_multicast: 21791
tx_crc_errors: 0
tx_undersize: 0
tx_oversize: 0
tx_fragment: 0
tx_jabber: 0
tx_collision: 0
tx_64byte: 21991
tx_65to127byte: 141
tx_128to255byte: 1
tx_256to511byte: 3
tx_512to1023byte: 1038
tx_1024to2047byte: 0
tx_GTE2048byte: 0
tx_octets: 1201607491
IEEE_tx_drop: 0
IEEE_tx_frame_ok: 23174
IEEE_tx_1col: 0
IEEE_tx_mcol: 0
IEEE_tx_def: 0
IEEE_tx_lcol: 0
IEEE_tx_excol: 0
IEEE_tx_macerr: 0
IEEE_tx_cserr: 0
IEEE_tx_sqe: 0
IEEE_tx_fdxfc: 21791
IEEE_tx_octets_ok: 1201607491
rx_packets: 29846
rx_broadcast: 10655
rx_multicast: 2247
rx_crc_errors: 0
rx_undersize: 0
rx_oversize: 0
rx_fragment: 0
rx_jabber: 0
rx_64byte: 10393
rx_65to127byte: 2425
rx_128to255byte: 510
rx_256to511byte: 46
rx_512to1023byte: 1745
rx_1024to2047byte: 14727
rx_GTE2048byte: 0
rx_octets: 3417714650
IEEE_rx_drop: 0
IEEE_rx_frame_ok: 29845
IEEE_rx_crc: 0
IEEE_rx_align: 0
IEEE_rx_macerr: 53964
IEEE_rx_fdxfc: 0
IEEE_rx_octets_ok: 3417713134

The interruption did not stop

281:     810435          0          0          0       GPC 118 Level     2188000.ethernet
282:          0          0          0          0       GPC 119 Level     2188000.ethernet

Any ideas on what could be causing this?

Will.

[庞磊]

The reception was normal at the beginning, but I tried to restore it after adding interference on the RX_CLK line, but this problem occurred, the data was not received, and rx_macerr was increasing. I cannot confirm whether this is the problem.

Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 1857771240 4115916 33 0 0 33 0 0 959174 1719 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0 0 680 8 0 0 0 0 0 0
can0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 1881159472 4131561 33 0 0 33 0 0 959764 1720 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0 0 680 8 0 0 0 0 0 0
can0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[ 1631.280175] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.285418] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.290650] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.295883] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.301115] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.306347] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.311578] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.316821] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.322055] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.327286] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.332516] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.337747] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.342977] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.348228] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.353460] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.358691] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.363921] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.369151] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.374380] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.379609] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.384838] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.390068] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.395318] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.400551] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.405793] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.411033] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.416265] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.421496] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.426737] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.431968] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.437198] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.442429] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.447659] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.452890] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.458119] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.463350] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.468580] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.473810] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.479039] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.484270] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.489499] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.494729] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.499959] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.505188] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.510418] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.515647] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.520876] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.526105] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.531343] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.536575] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.541806] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.547037] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.552267] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.557498] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.562728] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.567958] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.573187] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.578417] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.583646] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.588876] fec 2188000.ethernet eth0: rcv is not +last
[ 1631.594105] fec 2188000.ethernet eth0: rcv is not +last
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 1886877592 4135386 98 0 0 98 0 0 959806 1721 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0 0 680 8 0 0 0 0 0 0
can0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 1886877592 4135386 98 0 0 98 0 0 959848 1722 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0 0 680 8 0 0 0 0 0 0
can0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 1886877592 4135386 98 0 0 98 0 0 959890 1723 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0 0 680 8 0 0 0 0 0 0
can0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 1886877592 4135386 98 0 0 98 0 0 959890 1723 0 0 0 0 0 0
eth1: 0 0 0 0 0 0 0 0 680 8 0 0 0 0 0 0
can0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Inter-| Receive | Transmit

[Sanket_Parekh]

Hello @庞磊 ,

I hope you are doing well.

->Please make sure in the initial case when the board is connected to pc and pc sends the ARP req , did board give response to it.
->Please make sure that whether the board goes into suspend mode or not.
->Please make sure whether tcp_keep_alive feature is enabled,because it will keep you connection alive even after the data transmission stops after 1 or more days. One can check this event on the  wire shark.
->Please make sure to check with the latest Linux image, one should always use the latest linux image that is recommended.

Thanks & Regards,

Sanket Parekh

[庞磊]

Hello 

    I used UDP transmission interval of 1ms to transmit 1400 bytes. When there was a problem, macerr kept increasing and the upper layer could not receive any RX data. At this time, arp of my board could be sent to PC normally, and RX reception of the board seemed to be suspended.

    I found this patch in the community https://lkml.org/lkml/2016/11/17/945 And in the kernel version 4.1.15 version of this patch, indeed no RX stop receiving phenomenon.Because the time of appearance is uncertain, I am not sure whether there is still a problem, the patch code does not adapt to my kernel, I do not know whether this modification is reasonable.

+static inline bool
+fec_enet_recover_rxq(struct fec_enet_private *fep, u16 queue_id)
+{
+int work_bit = (queue_id == 0) ? 2 : ((queue_id == 1) ? 0 : 1);

+if(readl(fep->hwp + FEC_R_DES_ACTIVE(queue_id)))
+return false;

+dev_notice_once(&fep->pdev->dev, "Recovered rx queue\n");

+fep->work_rx |= 1 << work_bit;

+return true;
+}

+static inline bool fec_enet_recover_rxqs(struct fec_enet_private *fep)
+{
+unsigned int q;
+bool ret = false;

+for (q = 0; q < fep->num_rx_queues; q++) {
+if (fec_enet_recover_rxq(fep, q))
+ret = true;
+}

+return ret;
+}

static int fec_enet_rx_napi(struct napi_struct *napi, int budget)
{
struct net_device *ndev = napi->dev;
struct fec_enet_private *fep = netdev_priv(ndev);
int pkts;

pkts = fec_enet_rx(ndev, budget);

fec_enet_tx(ndev);

if (pkts < budget) {
napi_complete(napi);
writel(FEC_DEFAULT_IMASK, fep->hwp + FEC_IMASK);

+if (fec_enet_recover_rxqs(fep) && napi_reschedule(napi)){
+writel(FEC_ENET_RXF | FEC_ENET_RXB, fep->hwp + FEC_IMASK);
+}

}
return pkts;
}

Can you give me some advice?

thanks

[Sanket_Parekh]

Hello @庞磊 ,

I hope you are doing well.

->Please check whether the system is not going into suspend mode or low-power mode,after no communication for long time.
->If possible , please check with the latest kernel , it is always recommended,and share the observation with the latest kernel.

Thanks & Regards,

Sanket Parekh