I am trying to bring up a custom board with SJA1105P Ethernet switch, but I'm encountering difficulties getting any Ethernet traffic.
System architecture:
CPU: iMX8MP
MAC Switch: 1105P, with the xMII0 port connected to iMX8MP.
PHYs: 4 x KSZ9031 connected to xMII1 - xMII3 ports of the sja1105 Ethernet switch.
Kernel: Linux kernel 5.15.71.
Issue:
Unable to ping any of the devices connected to the sja1105P switch.
I've probed the MIIx_TXDx lines, there is no activity on any of the MIIx_TXDx output pins of the sja1105 switch. Unfortunately, I do not have access to xMII0 RX/TX lines to verify data send between CPU and the switch.
The sja1105P is recognized, and all KS9031 chips are detected. The Ethernet link is detected when Ethernet cable is connected. All MII1_TX_CLK - MII4_TX_CLK clocks are present.
Kernel output log:
[ 2.348721] sja1105 spi2.0: Probed switch chip: SJA1105P
[ 2.806647] sja1105 spi2.0: configuring for fixed/rgmii link mode
[ 2.813619] sja1105 spi2.0: Link is Up - 1Gbps/Full - flow control off
[ 2.867336] sja1105 spi2.0 swp0 (uninitialized): PHY [30be0000.ethernet-1:00] driver [Micrel KSZ9031 Gigabit PHY] (irq=202)
[ 2.947068] sja1105 spi2.0 swp1 (uninitialized): PHY [30be0000.ethernet-1:02] driver [Micrel KSZ9031 Gigabit PHY] (irq=204)
[ 2.984856] hub 3-1:1.0: USB hub found
[ 2.988977] hub 3-1:1.0: 4 ports detected
[ 3.019044] sja1105 spi2.0 swp2 (uninitialized): PHY [30be0000.ethernet-1:01] driver [Micrel KSZ9031 Gigabit PHY] (irq=200)
[ 3.095071] sja1105 spi2.0 swp3 (uninitialized): PHY [30be0000.ethernet-1:03] driver [Micrel KSZ9031 Gigabit PHY] (irq=206)
[ 3.108481] device eth0 entered promiscuous mode
[ 3.113149] DSA: tree 0 setup
[ 6.784978] Generic PHY fixed-0:00: attached PHY driver (mii_bus:phy_addr=fixed-0:00, irq=POLL)
[ 6.796261] fec 30be0000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[ 6.805022] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 6.815230] sja1105 spi2.0 swp0: configuring for phy/rgmii-id link mode
[ OK ] Found device /dev/mmcblk2p1.
[ 6.829732] sja1105 spi2.0 swp1: configuring for phy/rgmii-id link mode
[ OK ] Started Network Configuration.
[ 6.882694] sja1105 spi2.0 swp2: configuring for phy/rgmii-id link mode
[ 6.894170] sja1105 spi2.0 swp3: configuring for phy/rgmii-id link mode
[ 18.584955] sja1105 spi2.0 swp0: Link is Up - 100Mbps/Full - flow control off
[ 18.592151] IPv6: ADDRCONF(NETDEV_CHANGE): swp0: link becomes ready
My DSA switch configuration (example for one port):
ip link set eth0 down
ip addr add 192.168.1.100/30 dev swp0
ip link set eth0 up
ip link set swp0 up
ifconfig output
eth0 Link encap:Ethernet HWaddr 4A:2C:CD:1D:2E:74
inet addr:169.254.212.161 Bcast:169.254.255.255 Mask:255.255.0.0
inet6 addr: fe80::482c:cdff:fe1d:2e74/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1504 Metric:1
RX packets:97 errors:0 dropped:0 overruns:0 frame:0
TX packets:106 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20934 (20.4 KiB) TX bytes:22885 (22.3 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:164 errors:0 dropped:0 overruns:0 frame:0
TX packets:164 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11244 (10.9 KiB) TX bytes:11244 (10.9 KiB)
swp0 Link encap:Ethernet HWaddr 4A:2C:CD:1D:2E:74
inet addr:192.168.1.100 Bcast:0.0.0.0 Mask:255.255.255.252
inet6 addr: fe80::482c:cdff:fe1d:2e74/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:97 errors:0 dropped:0 overruns:0 frame:0
TX packets:45 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:18606 (18.1 KiB) TX bytes:9114 (8.9 KiB)
ethtool swp0
Settings for swp0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Full
100baseT/Full
1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Full
100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 100Mb/s
Duplex: Full
Auto-negotiation: on
master-slave cfg: preferred slave
master-slave status: master
Port: Twisted Pair
PHYAD: 0
Transceiver: external
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Link detected: yes
My dts settings:
&ecspi2 {
#address-cells = <1>;
#size-cells = <0>;
fsl,spi-num-chipselects = <1>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_ecspi2 &pinctrl_ecspi2_cs>;
cs-gpios = <&gpio5 13 GPIO_ACTIVE_LOW>;
status = "okay";
switch@0 {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_switch>;
#size-cells = <0>;
#address-cells = <2>;
compatible = "nxp,sja1105p";
reg = <0>;
spi-max-frequency = <4000000>;
spi-rx-delay-us = <1>;
spi-tx-delay-us = <1>;
spi-cpha;
status = "okay";
reset-gpios = <&gpio2 0 GPIO_ACTIVE_LOW>;
clocks = <&clock_sja1105>;
ports {
#address-cells = <1>;
#size-cells = <0>;
/* Implicit "sja1105,role-phy;" */
port@0 {
reg = <0>;
label = "cpu";
ethernet = <&fec>;
phy-mode = "rgmii-id";
//eee-broken-1000t;
rx-internal-delay-ps = <2000>;
tx-internal-delay-ps = <2000>;
//sja1105,role-phy;
fixed-link {
speed = <1000>;
full-duplex;
};
};
port@1 {
reg = <1>;
label = "swp0";
phy-handle = <ðphy1>;
phy-mode = "rgmii-id";
};
port@2 {
reg = <2>;
label = "swp1";
phy-handle = <ðphy2>;
phy-mode = "rgmii-id";
};
port@3 {
reg = <3>;
label = "swp2";
phy-handle = <ðphy3>;
phy-mode = "rgmii-id";
};
port@4 {
reg = <4>;
label = "swp3";
phy-handle = <ðphy4>;
phy-mode = "rgmii-id";
};
};
};
};
&fec {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_fec>;
phy-mode = "rgmii-id";
compatible = "fsl,imx8mp-fec", "fsl,imx8mq-fec", "fsl,imx6sx-fec";
status = "okay";
fixed-link {
speed = <1000>;
full-duplex;
};
mdio {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";
/* sja1105P port xMII1 */
ethphy1: ethernet-phy@1 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <0>;
interrupt-parent = <&gpio4>;
interrupts = <31 IRQ_TYPE_EDGE_FALLING>;
reset-gpios = <&gpio5 2 GPIO_ACTIVE_LOW>;
};
/* sja1105P port xMII2 */
ethphy2: ethernet-phy@2 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <2>;
interrupt-parent = <&gpio5>;
interrupts = <1 IRQ_TYPE_EDGE_FALLING>;
reset-gpios = <&gpio5 0 GPIO_ACTIVE_LOW>;
};
/* sja1105P port xMII3 */
ethphy3: ethernet-phy@3 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <1>;
interrupt-parent = <&gpio4>;
interrupts = <29 IRQ_TYPE_EDGE_FALLING>;
reset-gpios = <&gpio4 28 GPIO_ACTIVE_LOW>;
};
/* sja1105P port xMII4 */
ethphy4: ethernet-phy@4 {
compatible = "ethernet-phy-ieee802.3-c22";
reg = <3>;
interrupt-parent = <&gpio5>;
interrupts = IRQ_TYPE_EDGE_FALLING>;
reset-gpios = <&gpio4 30 GPIO_ACTIVE_LOW>;
};
};
};
How do I configure the SJA1105P switch? I've tried different DSA configurations (with/without tagging support), but I was unable to ping.
I have also attempted the test recommended in the post: https://community.nxp.com/t5/Other-NXP-Products/SJA1105Q-cannot-forward-eth-packages-form-the-rgmii-...
ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1504 qdisc mq state UP group default qlen 1000
link/ether 00:0a:35:00:22:02 brd ff:ff:ff:ff:ff:ff
inet6 fe80::20a:35ff:fe00:2202/64 scope link
valid_lft forever preferred_lft forever
3: swp0@eth0: <NO-CARRIER,BROADCAST,MULTICAST,DYNAMIC,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
link/ether 00:0a:35:00:22:02 brd ff:ff:ff:ff:ff:ff
4: swp1@eth0: <NO-CARRIER,BROADCAST,MULTICAST,DYNAMIC,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
link/ether 00:0a:35:00:22:02 brd ff:ff:ff:ff:ff:ff
5: swp2@eth0: <NO-CARRIER,BROADCAST,MULTICAST,DYNAMIC,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
link/ether 00:0a:35:00:22:02 brd ff:ff:ff:ff:ff:ff
6: swp3@eth0: <NO-CARRIER,BROADCAST,MULTICAST,DYNAMIC,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
link/ether 00:0a:35:00:22:02 brd ff:ff:ff:ff:ff:ff
ip link set eth0 up && ip link set swp0 up
ethtool -S eth0 | grep -v ': 0'
tx_packets: 65
tx_broadcast: 34
tx_multicast: 31
tx_64byte: 16
tx_65to127byte: 31
tx_256to511byte: 6
tx_512to1023byte: 12
tx_octets: 12936
IEEE_tx_frame_ok: 65
IEEE_tx_octets_ok: 12936
rx_packets: 178
rx_broadcast: 75
rx_multicast: 103
rx_65to127byte: 50
rx_128to255byte: 58
rx_256to511byte: 70
rx_octets: 48806
IEEE_rx_frame_ok: 178
IEEE_rx_octets_ok: 48806
p00_n_txfrm: 178
p00_n_txbyte: 48806
p00_n_tx_bytes_256_511: 70
p00_n_tx_bytes_128_255: 58
p00_n_tx_bytes_65_127: 50
p00_n_tx_mcast: 103
p00_n_tx_bcast: 75
ip addr add 192.168.100.1/24 dev swp0 && ping 192.168.100.2
ethtool -S eth0 | grep -v ': 0'
tx_packets: 31
tx_broadcast: 10
tx_multicast: 21
tx_65to127byte: 21
tx_256to511byte: 4
tx_512to1023byte: 6
tx_octets: 6866
IEEE_tx_frame_ok: 31
IEEE_tx_octets_ok: 6866
rx_packets: 7
rx_multicast: 7
rx_65to127byte: 2
rx_128to255byte: 5
rx_octets: 1229
IEEE_rx_frame_ok: 7
IEEE_rx_octets_ok: 1229
p00_n_txfrm: 8
p00_n_txbyte: 1297
p00_n_tx_bytes_128_255: 5
p00_n_tx_bytes_65_127: 3
p00_n_tx_mcast: 8
The values for p00_n_rxfrm and p00_n_rxbyte are both zero. I have not observed any increments in errors such as "N_RUNT", "N_SOFERR", "N_ALIGNERR", or "N_MIIERR".
Does this mean that there are no data being sent from the CPU to the switch? How can I verify if the switch received data? Unfortunately, I can't probe the xMII0 signals between the CPU and the switch.
Any suggestion?
Hello Petr,
Do you have any updates on my question regarding the performance/throughput of SJA1105? This has become a critical issue.
HI,
I asked AE team to comment this performance question.
Lets wait for feedback. Thanks for understandings.
BR, Petr
Hi,
this is response I got...
We need to have the configuration data of the device. To export the configuration into XML, you can use this tool from github: https://github.com/vladimiroltean/sja1105-tool Please provide this information.
Additionally, it would be beneficial to know the exact iperf command you use (to know what kind of traffic you sent) for inserting traffic to all 4 ports (I assume they refers to MII1 to MII4). Last but not least, can you check the diagnostic counters? In case there is traffic dropped, it might give a clue to why this is dropped.
BR, Petr
Hello Petr,
The sja1105 tool doesn't work with the kernel version 5.15.71-5.15.71-2.2.0 that I'm using.
I made some modifications to the sja1105-tool and was able to compile it for aarch64, but I'm encountering numerous errors:
[ 1349.750083] SPI driver sja1105 has no spi_device_id for nxp,sja1105e
[ 1349.756522] SPI driver sja1105 has no spi_device_id for nxp,sja1105t
[ 1349.762940] SPI driver sja1105 has no spi_device_id for nxp,sja1105p
[ 1349.769323] SPI driver sja1105 has no spi_device_id for nxp,sja1105q
[ 1349.775714] SPI driver sja1105 has no spi_device_id for nxp,sja1105r
[ 1349.782117] SPI driver sja1105 has no spi_device_id for nxp,sja1105s
[ 1349.813756] sja1105 spi2.0: Staging area node not present in device tree, trying sja1105.bin
[ 1349.822272] sja1105 spi2.0: Invalid port node in device tree
Is there a different mechanism/tool available to dump the switch configuration?
Regarding the iperf3 tool, I used the following commands:
iperf3 -s --port 5201 &
iperf3 -s --port 5202 &
iperf3 -s --port 5203 &
iperf3 -s --port 5204 &
iperf3 -c 192.168.1.36 --port 5201 -t 0
iperf3 -c 192.168.1.37 --port 5202 -t 0
iperf3 -c 192.168.1.38 --port 5203 -t 0
iperf3 -c 192.168.1.39 --port 5204 -t 0
The total throughput for Ethernet switch ports is always 100Mbps, regardless of whether I use 1, 2, or four ports (MII1 to MII4).
The ethtool shows that port 0 (MII0) is set to 1000Mbps:
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 1000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Auto-negotiation: on
Port: MII
PHYAD: 0
Transceiver: external
Supports Wake-on: d
Wake-on: d
Link detected: yes
Ports 1 to 4 (MII1-MII4) are configured to 100Mbps (example for port1):
Settings for swp0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Full
100baseT/Full
1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Full
100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 100Mb/s
Duplex: Full
Auto-negotiation: on
master-slave cfg: preferred slave
master-slave status: master
Port: Twisted Pair
PHYAD: 0
Transceiver: external
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Link detected: yes
I have captured some statistics during the test, but I do not find any obvious errors:
ethtool -S swp1 | grep -v ': 0'
NIC statistics:
tx_packets: 11603
tx_broadcast: 106
tx_multicast: 146
tx_64byte: 109
tx_65to127byte: 11454
tx_512to1023byte: 40
tx_octets: 10585035
IEEE_tx_frame_ok: 11603
IEEE_tx_fdxfc: 1
IEEE_tx_octets_ok: 10585035
rx_packets: 45995
rx_broadcast: 783
rx_multicast: 1128
rx_65to127byte: 997
rx_128to255byte: 466
rx_256to511byte: 351
rx_512to1023byte: 214
rx_1024to2047byte: 43967
rx_octets: 1364059196
IEEE_rx_frame_ok: 45995
IEEE_rx_octets_ok: 1364059196
p00_n_n664err: 64
p00_n_rxfrm: 142672
p00_n_rxbyte: 10584207
p00_n_txfrm: 897980
p00_n_txbyte: 1364085070
p00_n_not_reach: 128
p00_n_drops_utag: 64
p00_n_tx_bytes_1024_2047: 895958
p00_n_tx_bytes_512_1023: 214
p00_n_tx_bytes_256_511: 351
p00_n_tx_bytes_128_255: 466
p00_n_tx_bytes_65_127: 997
p00_n_tx_mcast: 1128
p00_n_tx_bcast: 783
p00_n_rx_bytes_512_1023: 39
p00_n_rx_bytes_65_127: 142527
p00_n_rx_bytes_64: 108
p00_n_rx_mcast: 192
ethtool -S swp0 | grep -v ': 0'
NIC statistics:
tx_packets: 39
tx_bytes: 6234
rx_packets: 539
rx_bytes: 103977
n_rxfrm: 539
n_rxbyte: 113679
n_txfrm: 39
n_txbyte: 6390
n_not_reach: 1017
n_tx_bytes_512_1023: 6
n_tx_bytes_65_127: 33
n_tx_mcast: 33
n_tx_bcast: 6
n_rx_bytes_512_1023: 12
n_rx_bytes_256_511: 132
n_rx_bytes_128_255: 122
n_rx_bytes_65_127: 150
n_rx_bytes_64: 123
n_rx_mcast: 318
ethtool -S swp1 | grep -v ': 0'
NIC statistics:
tx_packets: 39
tx_bytes: 6234
rx_packets: 557
rx_bytes: 105304
n_rxfrm: 557
n_rxbyte: 115330
n_txfrm: 39
n_txbyte: 6390
n_not_reach: 1077
n_tx_bytes_512_1023: 6
n_tx_bytes_65_127: 33
n_tx_mcast: 33
n_tx_bcast: 6
n_rx_bytes_512_1023: 12
n_rx_bytes_256_511: 132
n_rx_bytes_128_255: 124
n_rx_bytes_65_127: 164
n_rx_bytes_64: 125
n_rx_mcast: 335
ethtool -S swp2 | grep -v ': 0'
NIC statistics:
tx_packets: 37
tx_bytes: 6094
rx_packets: 861787
rx_bytes: 1291873089
n_crcerr: 1
n_rxfrm: 861798
n_rxbyte: 1307403471
n_txfrm: 37
n_txbyte: 6242
n_not_reach: 1106
n_tx_bytes_512_1023: 6
n_tx_bytes_65_127: 31
n_tx_mcast: 31
n_tx_bcast: 6
n_rx_bytes_1024_2047: 861194
n_rx_bytes_512_1023: 12
n_rx_bytes_256_511: 132
n_rx_bytes_128_255: 131
n_rx_bytes_65_127: 206
n_rx_bytes_64: 130
n_rx_mcast: 336
ethtool -S swp3 | grep -v ': 0'
NIC statistics:
tx_packets: 405590
tx_bytes: 26776337
rx_packets: 296434
rx_bytes: 443878630
n_rxfrm: 296448
n_rxbyte: 449235694
n_txfrm: 405596
n_txbyte: 28399435
n_not_reach: 1133
n_tx_bytes_512_1023: 4
n_tx_bytes_65_127: 405550
n_tx_bytes_64: 45
n_tx_mcast: 32
n_tx_bcast: 5
n_rx_bytes_1024_2047: 295861
n_rx_bytes_512_1023: 13
n_rx_bytes_256_511: 133
n_rx_bytes_128_255: 130
n_rx_bytes_65_127: 163
n_rx_bytes_64: 148
n_rx_mcast: 342
Hi,
feedback from Apps engineer...
I checked the default static configuration done by the linux DSA kernel driver and it uses a policer for each port with a max. bandwidth configured to be 1000Mbps. So this should be okay. However, the test setup is still not 100% clear to me. Attached I drew a block diagram of how I understand the customers topology. Can you ask if my understanding is correct?
I got a response from the creator of the sja1105-tool and he mentioned that another tool to dump the static configuration of the sja1105 is available here:https://github.com/vladimiroltean/mv88e6xxx_dump/tree/sja1105 With "devlink dev" they can find the name of the SPI device that you want to talk to, like "spi/spi0.1" (0 is bus, 1 is chip select). Then they can use a command like "./sja1105_dump --device spi/spi0.1 --static-config" to dump the static config devlink region from the kernel driver, and dump it as text output.
Lastly, whats the throughput if they use the following iperf command on client side:
iperf3 --udp -b 0 -c IP.ADDR -p PORT
BR, Petr
Hi,
could you please share your python script to see full switch configuration? Also share relevant schematic for connection between MCU, switch and PHYs.
The MCU MAC should have statistic counters as well, so you can refer to it.
Are you able to exchange ping/packets between xMII1 - xMII3 ports?
BR, Petr
Hi,
I'm experiencing an issue with the performance of the SJA1105 switch. The MII0 interface speed (the interface between the SJA1105 and the iMX8MP) is fixed at 1Gbps, while MII1 through MII4 are set to 100Mbps.
I conducted performance tests using iperf3 simultaneously across all four ports of the SJA1105 switch. However, I noticed inconsistent performance levels across these ports. Strangely, the total performance across all four ports is capped at 100Mbps, despite expecting each port to achieve 100Mbps individually.
I only observe 100Mbps throughput when a single port on the switch is used.
Could you provide any suggestions for identifying the problem and debugging it?
Hello,
I'm not using a Python script to configure the sja1105 switch. Instead, I'm relying on Linux DSA and the default static configuration provided by the sja1105 kernel driver
I managed to get the switch working. The issue stemmed from the imx-sdma driver not being ready (firmware not loaded) when accessed by the sja1105 driver. The solution was to utilize the sja1105 driver as a loadable kernel module and load it after the imx-sdma driver.
However, I've encountered a different problem now. The maximum speed I'm achieving on ports 1 to 4 is approximately 25 Mbps, whereas the ports should be running at 100 Mbps.
Do you have any suggestions on how to debug this issue?