IMX6ULL + KSZ8567: packet drop at 100mbps

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

IMX6ULL + KSZ8567: packet drop at 100mbps

1,148 Views
AaronElijah
Contributor II

Hello,

My team and I are trying to bringup a managed switch that uses the IMX6ULL as the host processor for the KSZ8567. Essentially, we've patched the DSA device driver for the KSZ9477 for the KSZ8567. See the patch file I've added to this question. We've also edited the device tree used by uboot and the linux kernel (using u-boot-imx and linux-imx) - see a snippet of the device tree below which describes the DSA topology.

 

 

&fec1 {
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_enet1>;
    phy-mode = "rmii";
    status = "okay";

    fixed-link {
        speed = <100>;
        full-duplex;
    };
};

&i2c1 {
    clock-frequency = <100000>;
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_i2c1>;
    status = "okay";

    switch@5f {
        compatible = "microchip,ksz8567";
        reg = <0x5f>;

        ports {
            #addess-cells = <1>;
            #size-cells = <0>;

			port@0 {
				reg = <0x00>;
				label = "lan1";
			};

			port@1 {
				reg = <0x01>;
				label = "lan2";
			};

			port@2 {
				reg = <0x02>;
				label = "lan3";
			};

			port@3 {
				reg = <0x03>;
				label = "lan4";
			};

			port@4 {
				reg = <0x04>;
				label = "lan5";
			};

			port@5 {
				reg = <0x05>;
				label = "cpu";
				ethernet = <&fec1>;
				phy-mode = "rmii";

				fixed-link {
					speed = <100>;
					full-duplex;
				};
			};
        };
    };
};

 

 

 

We setup a Linux bridge with the following commands.

ip link set eth0 up
ip link set lan5 up
ip link set lan4 up
ip link set lan3 up
ip link set lan2 up
ip link set lan1 up
ip link add name br0 type bridge vlan_filtering 1
ip link set dev lan5 master br0 
ip link set dev lan4 master br0 
ip link set dev lan3 master br0 
ip link set dev lan2 master br0 
ip link set dev lan1 master br0 
ip addr add 10.1.1.100/24 dev br0
ip link set dev br0 up 


However we are seeing from ping tests from two hosts connected on each of the switch ports that they occasionally drop packets. In the following image, you can see that Seq 41 ICMP echo is dropped.
ping_test.png

 

The same setup results in iperf3 TCP test which shows no packet transfer.

iperf_tcp_test.png

 

Likewise for an iperf UDP test, we can see 40% packet drop at 100Mbps.
iperf_udp_test.png

 

Ultimately we've tried the following:
1. Manually set offloaded bridge forwarding database entires
2. Turning off auto-negotiation and force setting speed to 100Mbps, full duplex. 
3. It is documented in KSZ8567 errata that EEE so we disabled that in all link partners.

Ultimately none of what we've tried helped so far.

My questions are:
1. Is the RMII backbone to the IMX6ULL problematic for the KSZ8567? Do we need to use a 1Gbit backbone for the CPU port instead?
2. Do we need to patch the fec controller in the linux-imx to work with KSZ switches?

Labels (1)
0 Kudos
7 Replies

940 Views
_angelo_
Contributor III

i understand, stp needs to mange ports, sure.

It can be enabled in the kernel, or implemented in userspace, as desired.

Good luck so, 

0 Kudos

1,078 Views
_angelo_
Contributor III

good luck !

sometimes, especially if you don't need rstp,  switches can also be configured with simple custom drivers, just seting up vlan ids for the configuration needed, without using dsa.

0 Kudos

1,033 Views
AaronElijah
Contributor II

Indeed and in fact I've done this before on the predecessor board for simple VLAN topologies. Unfortunately, we are interested in features that will require the host processor to access the data plane (STP being one of them). I'll post on this thread when/if we've made progress on this issue.

Tags (1)
0 Kudos

1,097 Views
_angelo_
Contributor III

dsa drivers are famous to be not very performant when passing through cpu port.

Sure, if it's a custom board, you need to check

- cpu port<->switch wiring, impedance matching and signal integrity (more important for 1Ghz), IO voltage levels (that can be set from imx side or either switch chip side) and mii delays that can be set from fdt
- bottleneck is 100mbit/s for rmii, so better to move to rgmii if you need full speed between cpu and ports (managed max speed)
- check cables, check against host pc or other devices.
- check updated drivers from mainline eventually

Angelo Dureghello

 

0 Kudos

1,081 Views
AaronElijah
Contributor II

Thanks for commenting Angelo!

Indeed DSA drivers for Microchip KSZ switches seem to have plenty of active development between the kernel 6.1 and 6.5 releases. At the moment, we're using the 6.1 linux-imx kernel (https://github.com/nxp-imx/linux-imx/tree/lf-6.1.y/drivers/net/dsa/microchip), which lacks commit from the 6.5 in mainline linux kernel (https://github.com/torvalds/linux/tree/v6.5/drivers/net/dsa/microchip). I'm sure that this will be remedied when NXP update their kernel to 6.5 release.

On your various points
- cpu<->switch wiring, impedance matching and signal integrity, etc.
We are currently testing with the oscilloscope to check the signal integrity. We're also testing changing the Drive Strength on the RMII pins on the IMX.

- bottleneck at 100mbit/s for RMII
I agree that this is an issue for full switch management (i.e. wanted to have fire-walling, etc). However, I don't think that congestion is really the issue on our board. We've used the ethtool to check the MIB counter on the KSZ8567 and that doesn't indicate any packet drop between the IMX and Switch host port. Also, logically, given that the KSZ9477 (the switch that the DSA driver intends to support) is a gbit switch and its port 6 backbone is RGMII, surely that would also have congestion issues on boards based on the KSZ9477 and yet designs still use the KSZ9477? Furthermore, wouldn't hardware offloading using SwitchDev API help alleviate much of the congestion (we hadn't configured the switch to forward all packets to the host port)?

- check cables
Good idea! We've actually used a variety of cables from UTP and Cat5 but seemingly still getting the same results

- check updated drivers
Definitely something I will try. As I say, the differences between the latest on the linux-imx branch and the mainline linux kernel with regards to Microchip DSA driver may be the solution here. 

I'll report back results of the checks above.

1,108 Views
jimmychan
NXP TechSupport
NXP TechSupport

Could you tell me which version of BSP are you using?

0 Kudos

1,100 Views
AaronElijah
Contributor II

Hi Jimmy - thanks for replying!

I'm using the i.MX Linux Yocto Project BSP 6.1.1_1.0.0 Release.

Basically part of my work was porting uboot-imx and linux-imx to our new board. As you can see from above, I've patched the DSA driver for Microchip KSZ switches in the linux-imx kernel to support the KSZ8567 switch. I've not made any changes to the FEC driver though.

0 Kudos