No tx on LS1046A ethernet (kernel 4.12)

[marco3181]

Hello,

For functional reasons, I had to go to kernel 4.12 version for my LS1046A RDB evaluation card.

I used a mix between the kernel.org arm-soc branch for-next (kernel/git/arm/arm-soc.git - ARM SoC tree ) as main source and the kernel.org qbman version (kernel/git/leo/linux.git - Leo kernel tree ) for the driver/soc/fsl/qbman/ files. I had to patch the Kconfig for qbman driver to permit ARM ([PATCH 10/10] fsl/qbman: Enable FSL_LAYERSCAPE config on ARM ) and used the fragment freescale.config from the freescale upstream branch ldup (http://git.freescale.com/git/cgit.cgi/ppc/upstream/linux.git/tree/?h=ldup ). I modified the ls1046a dts files to add missing configurations (mainly pcie) (see joined tar file). The rest of the system is based on the LSDK 2.0 1703. I used the cryptodev from the qoriq open source (GitHub - qoriq-open-source/cryptodev-linux ) based on 1.9 version and remove all reference to dpaa user part and pkc-host (problem with includes).

The system is working, I have my Ethernets. But there are 2 problems:

- Uboot is wrongly modifying the dts MAC address. It begins with the first Ethernet which is disabled on the RDB card, so all MAC address are shifted and wrong. But this is not my real problem. I will look later with the denx qoriq version (git.denx.de Git - u-boot-fsl-qoriq.git/summary ) based on the 2017.06.

- The Ethernet port is receiving frames but sent frame are not going out. A tcpdump on the other side can't see anything. On the card side, ifconfig statistic show send tx data. This is my problem.

My question is: what can I do to debug this? I don't know where to look. I know this not the LSDK version of the kernel and I had to remove some parts of the base image. But I don't know what is wrong with this asymetrical problem.

Thank you.

Marc

Original Attachment has been moved to: u-boot.txt.zip

Original Attachment has been moved to: freescale_dts_1046ardb.tar.zip

Original Attachment has been moved to: linux.txt.zip

[mcbridematt]

I recently came across this issue again working with CentOS, despite having no issues with other distributions with the "mainline" DPAA driver.

It turns out that some enterprise distributions enable irqbalance by default, and irqbalance will attempt to reassign the QMan interrupts away from their original CPUs.

If the QMan portal interrupt is assigned to another CPU then the Ethernet ports will cease to work.

To fix this issue, disable the irqbalance service.

QMan portal N should be serviced by the N'th CPU, as seen in /proc/interrupts:

 25:   10434111          0          0          0     GICv2 204 Level     QMan portal 0
 26:          0    4193264          0          0     GICv2 206 Level     QMan portal 1
 27:          0          0    4134958          0     GICv2 208 Level     QMan portal 2
 28:          0          0          0    4075946     GICv2 210 Level     QMan portal 3

[marco3181 · Accepted solution]

Hi,

I will close this problem. I have tested the Linux kernel 4.13.5 vanilla with some patches on the dpaa/fman parts and all is ok now.

Last time I checked, NXP Linux upstream 4.13 is still missing the fman/dpaa parts and the .dts configuration files.

Thank everyone for your help.

Marc

[mcbridematt]

Hi Marc,

Have you had any success with the ldup tree? A new version was posted last week: http://git.freescale.com/git/cgit.cgi/ppc/upstream/linux.git/log/?h=ldup 

On the LS1043A it works, but only under a LEDE/OpenWRT image, under Ubuntu (17.04) it quickly fails:

root@ubuntu:~# iperf3 -c 192.168.0.180
Connecting to host 192.168.0.180, port 5201
[ 4] local 192.168.0.143 port 40440 connected to 192.168.0.180 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 64.7 MBytes 543 Mbits/sec 0 63.6 KBytes
[ 4] 1.00-2.00 sec 20.4 MBytes 171 Mbits/sec 0 1.41 KBytes
[ 4] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 4] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
[ 4] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
^C[ 4] 5.00-5.62 sec 0.00 Bytes 0.00 bits/sec 0 1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-5.62 sec 85.1 MBytes 127 Mbits/sec 0 sender
[ 4] 0.00-5.62 sec 0.00 Bytes 0.00 bits/sec receiver

I'm guessing it might be due to different sysctl (I've tried using the LEDE sysctl on Ubuntu) or perhaps other settings applied by systemd - ubuntu iperf3 under chroot works fine. I'm going to try some other distributions (Debian, Gentoo) etc. just in case.

[marco3181]

I have to add that only SGMII port are totaly functionning (port 5-6). Only RGMII are not working fully (port 3-4).

For the 10G copper and Fiber (port 9-10), I could'nt try them.

[yipingwang]

Hello Marc LEFORESTIER,

Does this RGMII problem persist?

If yes, please provide u-boot and Linux Kernel booting up log.

Have a great day,
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[marco3181]

Hi,

Yes, still a problem with RGMII on the 4.12 kernel. Since SGMIIs were working, I used them for my work.

U-boot SGMII are working, only Linux SGMII are troublesomes.

I added the logs in the joined files of the main message.

I know, there are some enhancements to make but I am currently searching why my DSA switch is working properly.

I hope this will help you to understand my problem.

Regards,

Marc

[yipingwang]

Hello Marc,

Please attach the Linux Kernel booting up log.

Thanks,

Yiping

[marco3181]

Hello,

The kernel booting up log is in the attachment files. It's the linux.txt file.

Marc