SJA1105Q: bridge mode not working

[fvalla-MTA]

Hello,

I am trying to use a SJA1105Q switch connected to an i.MX8QM processor (on port 4) and to three Broadcom PHYs (on ports 0-1-2). I am using Linux 5.10.71 as found in i.MX SDK release Linux 5.15.71_2.2.1, and the SJA1105 driver included in the kernel.

I set up a bridge using systemd (but I also tried manually) to be able to use the switch in "bridge mode", as indicated in https://docs.kernel.org/networking/dsa/configuration.html

I am however not able to communicate neither from my board to an external entity nor between entities connected to two ports of the switch.

Here is my configuration:

devicetree

&fec1 {
	status = "okay";
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_fec1>;

	/* Clock delay on TXC and RXC lines are applied at switch side */
	phy-mode = "rgmii-id";

	nvmem-cells = <&fec_mac0>;
	nvmem-cell-names = "mac-address";

	phy-reset-gpios = <&lsio_gpio0 26 GPIO_ACTIVE_HIGH>;
	phy-reset-duration = <1>;
	phy-reset-post-delay = <1>;

	mdio {
		#address-cells = <1>;
		#size-cells = <0>;

		fec1_phy0: ethernet-phy@0 {
			compatible = "ethernet-phy-ieee802.3-c45";
			reg = <0>;
		};

		fec1_phy1: ethernet-phy@1 {
			compatible = "ethernet-phy-ieee802.3-c45";
			reg = <1>;
		};
	};

	fixed-link {
		speed = <1000>;
		full-duplex;
	};
};

&lpspi0 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_lpspi0 &pinctrl_lpspi0_cs>;
	status = "okay";

	cs-gpios = <&lsio_gpio3 5 GPIO_ACTIVE_LOW>;

	sja1105: ethernet-switch@0 {
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_eth_hub>;
		compatible = "nxp,sja1105q";
		#address-cells = <1>;
		#size-cells = <0>;

		reg = <0>;
		spi-max-frequency = <12000000>;

		/* Sample data on trailing clock edge */
		spi-cpha;

		/* SPI controller settings for SJA1105 timing requirements */
		fsl,spi-cs-sck-delay = <1000>;
		fsl,spi-sck-cs-delay = <1000>;

		/* Driver reset logic is inverted - fix it with GPIO_ACTIVE_LOW here */
		reset-gpios = <&lsio_gpio0 16 GPIO_ACTIVE_LOW>;
		sja1105,startup-delay = <5>;

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				label = "swp0";
				phy-handle = <&fec1_phy0>;
				/* Clock delays are applied by the external PHY */
				phy-mode = "rgmii-id";
				reg = <0>;
			};

			port@1 {
				label = "swp1";
				phy-handle = <&fec1_phy1>;
				/* Clock delays are applied by the external PHY */
				phy-mode = "rgmii-id";
				reg = <1>;
			};

			port@2 {
				label = "swp2";
				phy-handle = <&fec2_phy0>;
				/* Clock delays are applied by the external PHY */
				phy-mode = "rgmii-id";
				reg = <2>;
			};

			port@4 {
				ethernet = <&fec1>;
				/* Clock delays are applied by the SJA1105 */
				phy-mode = "rgmii-id";
				reg = <4>;

				fixed-link {
					speed = <1000>;
					full-duplex;
				};
			};
		};
	};
};

br0.netdev

[NetDev]
Name=br0
Kind=bridge

br0.network

[Match]
Name=br0

[Network]
Address=192.168.0.2/24

swp.network

[Match]
Name=swp*

[Network]
BindCarrier=eth0
Bridge=br0
ConfigureWithoutCarrier=yes

Output of ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 qdisc mq state UP group default qlen 1000
    link/ether ba:01:11:86:31:72 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::b801:11ff:fe86:3172/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether ea:23:fd:7f:ac:e2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.2/24 brd 192.168.0.255 scope global eth1
       valid_lft forever preferred_lft forever
4: can0: <NOARP,UP,LOWER_UP,ECHO> mtu 16 qdisc pfifo_fast state UP group default qlen 10
    link/can 
5: can1: <NOARP,UP,LOWER_UP,ECHO> mtu 16 qdisc pfifo_fast state UP group default qlen 10
    link/can 
6: swp0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether ba:01:11:86:31:72 brd ff:ff:ff:ff:ff:ff
7: swp1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether ba:01:11:86:31:72 brd ff:ff:ff:ff:ff:ff
8: swp2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether ba:01:11:86:31:72 brd ff:ff:ff:ff:ff:ff
9: can2: <NOARP,UP,LOWER_UP,ECHO> mtu 16 qdisc pfifo_fast state UP group default qlen 10
    link/can 
10: usb0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 12:0c:32:b5:90:87 brd ff:ff:ff:ff:ff:ff
11: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:01:11:86:31:72 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.100/12 brd 172.31.255.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::dcd9:3aff:fec2:1ec8/64 scope link 
       valid_lft forever preferred_lft forever

Output of bridge link

6: swp0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4 
7: swp1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 19 
8: swp2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 19

Do you have any idea on why this is happening?

Thank you very much in advance

Kind regards,

Francesco Valla

[fvalla-MTA · Accepted solution]

Dear @brian14 (and all other concerned parties),

I found the issue to be an incorrect configuration of the RGMII delays inside the driver of the PHY, so not really an issue of the SJA1105Q.

As a reference, my final DTS configuration is the following one (in the end, I decided to apply RGMII delays at SoC and PHYs side):

&fec1 {
	status = "okay";
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_fec1>;

	/* Clock delay on TXC and RXC lines are applied at FEC side by the
	* fsl,rgmii_*xc_dly properties below */
	phy-mode = "rgmii";
	fsl,rgmii_rxc_dly;
	fsl,rgmii_txc_dly;

	nvmem-cells = <&fec_mac0>;
	nvmem-cell-names = "mac-address";

	phy-reset-gpios = <&lsio_gpio0 26 GPIO_ACTIVE_HIGH>;
	phy-reset-duration = <1>;
	phy-reset-post-delay = <1>;

	mdio {
		#address-cells = <1>;
		#size-cells = <0>;

		fec1_phy0: ethernet-phy@0 {
			compatible = "ethernet-phy-ieee802.3-c45";
			reg = <0>;
		};

		fec1_phy1: ethernet-phy@1 {
			compatible = "ethernet-phy-ieee802.3-c45";
			reg = <1>;
		};
	};

	fixed-link {
		speed = <1000>;
		full-duplex;
	};
};

&lpspi0 {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_lpspi0 &pinctrl_lpspi0_cs>;
	status = "okay";

	cs-gpios = <&lsio_gpio3 5 GPIO_ACTIVE_LOW>;

	sja1105: ethernet-switch@0 {
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_eth_hub>;
		compatible = "nxp,sja1105q";
		#address-cells = <1>;
		#size-cells = <0>;

		reg = <0>;
		spi-max-frequency = <12000000>;

		/* Sample data on trailing clock edge */
		spi-cpha;

		/* SPI controller settings for SJA1105 timing requirements */
		fsl,spi-cs-sck-delay = <1000>;
		fsl,spi-sck-cs-delay = <1000>;

		/* Driver reset logic is inverted - fix it with GPIO_ACTIVE_LOW here */
		reset-gpios = <&lsio_gpio0 16 GPIO_ACTIVE_LOW>;
		sja1105,startup-delay = <5>;

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				label = "swp0";
				phy-handle = <&fec1_phy0>;
				/* Clock delays are applied by the external PHY */
				phy-mode = "rgmii-id";
				reg = <0>;
			};

			port@1 {
				label = "swp1";
				phy-handle = <&fec1_phy1>;
				/* Clock delays are applied by the external PHY */
				phy-mode = "rgmii-id";
				reg = <1>;
			};

			port@2 {
				label = "swp2";
				phy-handle = <&fec2_phy0>;
				/* Clock delays are applied by the external PHY */
				phy-mode = "rgmii-id";
				reg = <2>;
			};

			port@4 {
				ethernet = <&fec1>;
				/* Clock delays are applied by the SoC */
				phy-mode = "rgmii";
				reg = <4>;

				fixed-link {
					speed = <1000>;
					full-duplex;
				};
			};
		};
	};
};

[brian14]

Thank you for contacting NXP Support and for your detailed information.

I have reviewed your configurations, and it seems that they are correct.

However, I think you can use a similar debugging process described by a colleague on the following thread:
The four phys connected to sja1105Q cannot be load... - NXP Community

Have a great day!