SJA1105 in Linux, all 5 ports through PHY interfaces?

[kalamata]

Hi,

I'm working on a project based around a SMARC format computer-on-a-module with a Rockchip RK3399 CPU. On the same motherboard we have a SJA1105 switch. The CPU runs Linux (5.4) and uses the NET_DSA_SJA1105 driver.

Our CPU needs to communicate with other devices over Ethernet so the CPU needs to be connected to the switch.

In the SJA1105 datasheet and in the Linux kernel device tree examples the host cpu is always connected to the switch as a fixed link directly over xMII. (https://www.kernel.org/doc/Documentation/devicetree/bindings/net/dsa/sja1105.txt)

We cannot do this as our SMARC module only gives us pin output from the RK3399 Ethernet integrated PHY, we do not have physical access directly to the xMII interface on the CPU no matter how much we would want to.

Also in the SJA1105 datasheet it says the following:

"Note that Ethernet connectivity to the host processor is only needed if the system has to support AVB operation or other bridge management protocols such as STP/RSTP. If such operations are not needed, all the ports can be used for data traffic."

Since we don't need AVB or STP/RSTP and since we anyway don't have access to the cpu xMII interface we have decided to just use the switch as a normal 5-port switch with PHY interfaces on every port. 4 ports where the PHY's actually goes to real RJ45 connectors and one port where the PHY is hardwired in the circuit board to the PHY that we get from our CPU on the SMARC board. This would give us a Linux device tree looking something like this:

(only SJA1105 relevant part shown, indentation also became screwed when pasted here)

&spi2 {
status = "okay";
#address-cells = <1>;
#size-cells = <0>;

num-cs = <2>;
cs-gpios = <&gpio2 RK_PB4 GPIO_ACTIVE_LOW>, <&gpio1 RK_PA3 GPIO_ACTIVE_LOW>;

sja1105_1: sja1105@1 {
status = "okay";
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
clocks = <&ethswitch_osc>;
compatible = "nxp,sja1105t";
spi-max-frequency = <25000000>;

fsl,spi-cs-sck-delay = <1000>;
fsl,spi-sck-cs-delay = <1000>;
spi-cpha;
pinctrl-names = "default";
pinctrl-0 = <&ethswitch_pins>;
reset-gpios = <&gpio4 RK_PC5 GPIO_ACTIVE_HIGH>;

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
/* Implicit "sja1105,role-mac;" */
label = "eth0";
phy-handle = <&rmii_phy0>;
phy-mode = "rmii";
reg = <0>;
};

port@1 {
/* Implicit "sja1105,role-mac;" */
label = "lan1";
phy-handle = <&rmii_phy1>;
phy-mode = "rmii";
reg = <1>;
};

port@2 {
/* Implicit "sja1105,role-mac;" */
label = "lan2";
phy-handle = <&rmii_phy2>;
phy-mode = "rmii";
reg = <2>;
};
port@3 {
/* Implicit "sja1105,role-mac;" */
phy-handle = <&rmii_phy3>;
label = "lan3";
phy-mode = "rmii";
reg = <3>;
};
port@4 {
/* Implicit "sja1105,role-mac;" */
phy-handle = <&rmii_phy4>;
label = "lan4";
phy-mode = "rmii";
reg = <4>;
};

};
};
};

This devicetree is valid and compiles, however during the probe the NET_DSA_SJA1105 fails because it cannot find a HOST CPU port (as we have not specified one). This error comes from net/dsa/dsa2.c *dsa_tree_find_first_cpu(struct dsa_switch_tree *dst) so it's not specific to sja1105 part of the driver but rather deeper in the Linux Distributed Switch Architecture.

In all devicetree examples there is always one port configured as host cpu connected with fixed link over xMII, for example like this:

port@4 {
/* Internal port connected to eth2 */
ethernet = <&enet2>;
phy-mode = "rgmii";
reg = <4>;
/* Implicit "sja1105,role-phy;" */
fixed-link {
speed = <1000>;
full-duplex;
};
};

If I try to modify my device tree to have the ports look something more like this:

ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
/* Implicit "sja1105,role-phy;" */
ethernet = <&gmac>;
phy-mode = "rmii";
reg = <0>;
fixed-link {
speed = <100>;
full-duplex;
};
};

port@1 {
/* Implicit "sja1105,role-mac;" */
label = "lan1";
phy-handle = <&rmii_phy1>;
phy-mode = "rmii";
reg = <1>;
};

port@2 {
/* Implicit "sja1105,role-mac;" */
label = "lan2";
phy-handle = <&rmii_phy2>;
phy-mode = "rmii";
reg = <2>;
};
port@3 {
/* Implicit "sja1105,role-mac;" */
phy-handle = <&rmii_phy3>;
label = "lan3";
phy-mode = "rmii";
reg = <3>;
};
port@4 {
/* Implicit "sja1105,role-mac;" */
phy-handle = <&rmii_phy4>;
label = "lan4";
phy-mode = "rmii";
reg = <4>;
};

};

Then the driver is satisfied and probes successfully and sets up the SJA1105 switch. Ports 1-4 work
but port 0 (where CPU is connected via a PHY) does of course not work as it's now configured as xMII would be directly connected to it when in reality we have a PHY there in between the SJA1105 and the host CPU.

Is this a limitation the the DSA SJA1105 Linux driver, that you always must configure a fixed link over xMII? Can I somehow configure directly from the device tree that i DO NOT wish to have a dedicated cpu host port at all but rather just 5 normal ports (of which one then happens to so be connected to the CPU via a PHY...) Or can I configure the CPU host port to be there as a fixed-link, but explicitly specify that there is actually a PHY there also and not a xMII interface? I have tried dozens and dozens of combinations and still not found any way to configure this so that it will work with our supposedly very simple setup. Am I just missing something really obviously simple here? Any ideas how to proceed? Or is this just not possible with the Linux DSA SJA1105 driver?

Regards

[RichieEng]

@kalamata @Sabeur were you able to arrive at a solution to this issue. I am running into a similar situation with Toradex imx8mp SoM. 

[RichieEng]

The PHY-to-PHY back to back is possible. You have to use the phy-handle property. You can also bypass auto-negotiation and set a fix speed with the "fixed-link" property in devicetree bindings. If you are having problem and you suspect your devicetree configuration is correct, kindly check the control path interface (SPI/I2C/MDIO).

[Sabeur]

Hi, 

I see the phy-mode and phy-handle for the port 0:

port@0 {
/* Implicit "sja1105,role-mac;" */

ethernet = <&gmac>;
label = "eth0";
phy-handle = <&rmii_phy0>;
phy-mode = "rmii";
reg = <0>;
};

This is how the device tree should look like.

Also, the gmac node should contain

&gmac {
    phy-handle = <&rmii_gmac_phy>;
    phy-mode = "rmii";
};

best regards,

Sabeur

[Sabeur]

Hi Kalamata, 

Thanks for the details. 

So my assumptions were correct. 

You should be able to specify phy-handle and phy-mode for the connected MAC interfaces. It was tested virtually, means MAC -> PHY - > --- <- PHY <- MAC from 5.3 Kernel version. (see 0e27921816ad99f78140e0c61ddf2bc515cc7e22).
So this is the right way to describe this kind of HW connection in the device tree.

Regards, 
Sabeur

[Sabeur]

Hi,

Theoretically it should be possible however, i will not work out of the box because the expected setup is to have a direct link.

The data path which can work is: Mac - Phy - Phy - Switch Mac.

In this case you will need to modify the DSA kernel and specify Phy handle and Phy mode for both CPU Mac and Switch Mac.

Now the question is: is the above data path possible in your setup ? Has the Rockchip RK3399 CPU an internal PHY ?

Best regards,
Sabeur

[kalamata]

Hi Sabeur,

Thank you for the reply!

Is the above data path possible in your setup ?
Yes, we are using exactly that setup: (CPU) Mac - Phy - Phy - Switch Mac
(see attachments)

Has the Rockchip RK3399 CPU an internal PHY ?
Yes, it has an internal PHY, and we are stuck using this internal PHY, as it's the only thing coming out from the Advantech ROM-5780 SMARC module where our CPU is located, so we could not use the CPU MAC interface directly even if we wanted to.

See attached pictures for how our setup differs from the typical setup suggested by the SJA1105 datasheet.

We have an Advantech ROM-5780 Computer-on-a-chip module (i.e. a Rockchip RK3399 CPU) packaged in SMARC format. The Rockchip RK3399 has an integrated PHY interface and that is what we have access to on the output pins on the SMARC module. We do NOT have physical access to the MAC pins of the RK3399 Ethernet controller as they are not coming out via the SMARC module, so we are stuck using the RK3399 PHY interface. Therefore on the 5:th port on the SJA1105 where we intend to connect our CPU we have a PHY controller (just like we have on ports 1-4) and that PHY controller is permanently wired to the PHY controller coming out of the RK3399 CPU.




Regards,
Kalamata