LS1088ARDB - Ethernet Physical Phy Not working

[mloder]

I can't get networking to work when the LS1088ARDB is configured to use the onboard physical phys.  I have switched to using the physical phys to work with a PoE driver which attaches to the phys in the device tree. To enable the physical phys I have done this by updating the DPC file to include.

	board_info {
		ports {
			mac@7 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
			mac@8 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
			mac@9 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
			mac@10 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
		};

 When I run ethtool dpmac7 the links report as 

Link detected: no

When I run mii-tool mii-tool dpmac7 -v it reports the link as up and appears to be communicating with the phy.

dpmac7: negotiated 1000baseT-FD, link ok
  product info: vendor 00:01:c1, model 39 rev 0
  basic mode:   autonegotiation enabled
  basic status: autonegotiation complete, link ok
  capabilities: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  advertising:  1000baseT-FD 100baseTx-FD 10baseT-FD flow-control
  link partner: 1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control

When I configure an IP for the port I can't get communications to flow either. I am not sure where to look next.

Any help would be appreciated

Thanks

Mike Loder

[yipingwang]

Please note, dpmac.7 is QSGMII5.

I used the default dpc and dpl configuration provided in LSDK.

root@TinyLinux:~# ls-listni
dprc.1/dpni.0 (interface: eth0, end point: dpmac.5)
root@TinyLinux:~# ls-listmac
dprc.1/dpmac.10
dprc.1/dpmac.9
dprc.1/dpmac.8
dprc.1/dpmac.7
dprc.1/dpmac.6
dprc.1/dpmac.5 (end point: dpni.0)
dprc.1/dpmac.4
dprc.1/dpmac.3
dprc.1/dpmac.2
dprc.1/dpmac.1
root@TinyLinux:~# ls-addni dpmac.7
[ 68.125523] iommu: Adding device dpbp.0 to group 7
[ 68.131472] iommu: Adding device dpmcp.26 to group 7
[ 68.136596] iommu: Adding device dpcon.8 to group 7
[ 68.141614] iommu: Adding device dpcon.7 to group 7
[ 68.146588] iommu: Adding device dpcon.6 to group 7
[ 68.151559] iommu: Adding device dpcon.5 to group 7
[ 68.156528] iommu: Adding device dpcon.4 to group 7
[ 68.161497] iommu: Adding device dpcon.3 to group 7
[ 68.166464] iommu: Adding device dpcon.2 to group 7
[ 68.171433] iommu: Adding device dpcon.0 to group 7
[ 68.176409] iommu: Adding device dpni.1 to group 7
[ 68.181853] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpmcp.2
[ 68.216790] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.8
[ 68.224950] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.8
[ 68.231494] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.7
[ 68.239602] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.7
[ 68.246112] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.6
[ 68.254276] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.6
[ 68.260819] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.5
[ 68.268921] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.5
[ 68.275432] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.4
[ 68.283593] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.4
[ 68.290148] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.3
[ 68.298248] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.3
[ 68.304759] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.2
[ 68.312922] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.2
[ 68.319474] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpcon.0
[ 68.327580] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpio.1
[ 68.334108] fsl_dpaa2_eth dpni.1: Linked as a consumer to dpbp.0
[ 68.381894] fsl_dpaa2_eth dpni.1: Probed interface eth1
Created interface: eth1 (object:dpni.1, endpoint: dpmac.7)
root@TinyLinux:~# ls-listni
dprc.1/dpni.1 (interface: eth1, end point: dpmac.7)
dprc.1/dpni.0 (interface: eth0, end point: dpmac.5)
root@TinyLinux:~# ifconfig eth1 100.1.1.100
[ 86.662760] fsl_dpaa2_eth dpni.1 eth1: Link Event: state up
root@TinyLinux:~# ping 100.1.1.1
PING 100.1.1.1 (100.1.1.1): 56 data bytes
64 bytes from 100.1.1.1: seq=0 ttl=64 time=0.465 ms
64 bytes from 100.1.1.1: seq=1 ttl=64 time=0.201 ms
64 bytes from 100.1.1.1: seq=2 ttl=64 time=0.211 ms
64 bytes from 100.1.1.1: seq=3 ttl=64 time=0.192 ms
^C
--- 100.1.1.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.192/0.267/0.465 ms
root@TinyLinux:~#

[mloder]

Hi yippingwang,

Thanks for the response and for giving details. You indicated that you used the default DPC file though. The DPC file needs to be modified like below to enable Phy Mode, not Fixed mode which the default is. I need to do this to allow linux to register and install the phys so I can work directly with the phys. The problem is when I do this I can't get ethernet traffic to flow. Are there other settings I need to make in the DPC file to make this work? Also is there somewhere the DPC config is documented? i can't find many details on it.

	board_info {
		ports {
			mac@7 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
			mac@8 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
			mac@9 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
			mac@10 {
				link_type = "MAC_LINK_TYPE_PHY";
			};
		};

Thanks

Mike

[yipingwang]

Please refer to the following mdio information, the default configuration is connecting to Vitesse VSC8514 PHY.

No need to modify DPC file, Linux Kernel will use Ethernet information provided in dts file arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts

=> mdio list
mdio@8B97000:
0 - Generic PHY <--> DPMAC2@xgmii
mdio@8B96000:
c - Vitesse VSC8514 <--> DPMAC3@qsgmii
d - Vitesse VSC8514 <--> DPMAC4@qsgmii
e - Vitesse VSC8514 <--> DPMAC5@qsgmii
f - Vitesse VSC8514 <--> DPMAC6@qsgmii
1c - Vitesse VSC8514 <--> DPMAC7@qsgmii
1d - Vitesse VSC8514 <--> DPMAC8@qsgmii
1e - Vitesse VSC8514 <--> DPMAC9@qsgmii
1f - Vitesse VSC8514 <--> DPMAC10@qsgmii
=>

[mloder]

Hi yipingwang,

Let me add some more context. I am currently attaching a TPS23881 PoE controller development board to the LS1088ARDB development board. I am in the process of getting the driver for this controller operational.

When I boot the development board up in default configuration Linux does not register any of the Phy devices which prevents the TPS23881 driver from starting. 

Default DPC - Linux (All ports report not supported, Linux can't access the MDIO phy devices)

# mii-tool eth3
SIOCGMIIPHY on 'eth3' failed: Operation not supported

If I update the DPC file to configure DPMAC7-10 to be 

link_type = "MAC_LINK_TYPE_PHY";

 Then Linux registers the PHY and also installs the drivers for them. See below.

[    7.347026] fsl_dpaa2_eth dpni.8 (unnamed net_device) (uninitialized): PHY [0x0000000008b96000:1f] driver [Microsemi GE VSC8514 SyncE] (irq=98)
[    7.362741] fsl_dpaa2_eth dpni.8: Probed interface eth0
[    7.607016] fsl_dpaa2_eth dpni.7 (unnamed net_device) (uninitialized): PHY [0x0000000008b96000:1e] driver [Microsemi GE VSC8514 SyncE] (irq=98)
[    7.622884] fsl_dpaa2_eth dpni.7: Probed interface eth1
[    7.867007] fsl_dpaa2_eth dpni.6 (unnamed net_device) (uninitialized): PHY [0x0000000008b96000:1d] driver [Microsemi GE VSC8514 SyncE] (irq=98)
[    7.882974] fsl_dpaa2_eth dpni.6: Probed interface eth2
[    8.127024] fsl_dpaa2_eth dpni.5 (unnamed net_device) (uninitialized): PHY [0x0000000008b96000:1c] driver [Microsemi GE VSC8514 SyncE] (irq=98)
[    8.143003] fsl_dpaa2_eth dpni.5: Probed interface eth3
[    8.267789] fsl_dpaa2_eth dpni.4: Probed interface eth4
[    8.392327] fsl_dpaa2_eth dpni.3: Probed interface eth5
[    8.516860] fsl_dpaa2_eth dpni.2: Probed interface eth6
[    8.641478] fsl_dpaa2_eth dpni.1: Probed interface eth7
[    8.767022] fsl_dpaa2_eth dpni.0: Probed interface eth8

You can see 4 of the ports now have VSC8514 drivers installed in the kernel, and I can successfully access the PHY with the mii-tool. Also the TPS23881 driver properly kicks in for the ports set for MAC_LINK_TYPE_PHY.

# mii-tool eth3
eth3: negotiated 100baseTx-FD flow-control, link ok

When I make this change however I can no longer get ethernet traffic to flow for example ETH3 can't reach the network. I am trying to determine if there is some other setting I need to make to configure the LINK_TYPE_PHY mode.

Also ethtool reports the phy is down, however the mii-tool above shows it connected.

# ethtool eth3                                                                                                                          
Settings for eth3:
        Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Full
                                100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Full
                                100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Auto-negotiation: on
        master-slave cfg: preferred master
        master-slave status: unknown
        Port: Twisted Pair
        PHYAD: 28
        Transceiver: external
        MDI-X: off (auto)
        Link detected: no

This is showing so far that I can't use the default DPC configuration. I am just stuck on how to get network traffic to work in this mode.

Thanks

Mike

[yipingwang]

You are indeed right that with the default DPC configuration, the PHYs are not probed and managed by Linux.

This is why nothing in the device-tree will be taken into account when the DPMAC is in TYPE_FIXED.

I just did a quick test on a LS1088ARDB and I can get a link up and traffic going while in TYPE_PHY with Linux kernel 6.6 (logs attached).

Could you please let me know what version of the Linux kernel are you using?

What is the link partner used for this test? Could you maybe test with two of the ports in an external loopback and see how it behaves?

Also, maybe it would be great if you also test with an unmodified Linux kernel image (without the TPS23881's driver) so that we can narrow down where is the problem.

Also, please keep in mind that the ETHx labels on the LS1088ARDB board do not match with the dpmac.y's index. Please make sure that you are using the correct interfaces.

[mloder]

We are using the Mickledore Yocto build for the LS1088ARDB - Kernel 6.1.55.

Linux localhost 6.1.55+g770c5fe2c1d1 #1 SMP PREEMPT Tue Nov 21 23:45:58 UTC 2023 aarch64 GNU/Linux

I made a build using the default kernel with no modifications from the NXP QoriQ Kernel `lf-6.1.y` branch.

I captured the full set of logs with the default DPC configuration and showed it establishing a network connection. Then rebuilding with the DPC changed to TYPE_PHY, showing no network connectivity.

I did notice that in both cases I had to delete the DPMAC5 configuration on boot and reconfigure all the ports to get the default to work. Just calling `ls-addni DPMAC7` did not work for some reason.

Thanks for the help

Yipingwang

[yipingwang]

Please refer to the following update from the SDK development team.

I didn't make any modifications to the kernel.

Could you please remove the 'interrupts-extended' field from the QSGMII PHYs DT node like below and re-test?

--- a/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts

+++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts

@@ -83,42 +83,34 @@ &emdio1 {

        status = "okay";

        mdio1_phy5: ethernet-phy@c {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0xc>;

        };

        mdio1_phy6: ethernet-phy@d {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0xd>;

        };

        mdio1_phy7: ethernet-phy@e {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0xe>;

        };

        mdio1_phy8: ethernet-phy@f {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0xf>;

        };

        mdio1_phy1: ethernet-phy@1c {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0x1c>;

        };

        mdio1_phy2: ethernet-phy@1d {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0x1d>;

        };

        mdio1_phy3: ethernet-phy@1e {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0x1e>;

        };

        mdio1_phy4: ethernet-phy@1f {

-               interrupts-extended = <&extirq 1 IRQ_TYPE_LEVEL_LOW>;

                reg = <0x1f>;

        };

 };

This is not presented as a fix but rather as another step in the debug procedure.

[mloder]

Hi Yippingwang,

This change worked, I removed the `interrupts-extended` as per your instructions and I am successfully getting link status reported through ethtool now, as well as I can properly establish a network link so this is great news.

You indicated this is not a fix to the issue, is there some addition information I can provide to get to a proper fix?

Again thank you for the continued support on this.

Mike

[yipingwang]

Confirming with the LSDK development team.