帮助
英语（美国） | English (US) 日本語 | Japanese 中文 | Chinese (Simplified)

Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction

取消
开启建议
自动建议可通过在您键入时建议可能的匹配，而快速缩小您的搜索结果范围。
显示结果 
显示  仅  | 搜索替代 
您的意思是: 

Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction

18 小时之前
78 次查看
pcentauri92
Contributor II

Board: Custom S32G399A based module, derived from S32G-VNP-RDB3. PFE_MAC1 connected via RGMII (PE_02–PE_13) to an NXP SJA1110A switch port 2, configured as the DSA CPU port (in-tree sja1105 driver, kernel 6.x BSP43.0).

Topology:
- PFE_MAC0: SGMII via SerDes1 lane1, Mode 1
- PFE_MAC1: RGMII to SJA1110A port 2 (DSA CPU port) — the port in question
- PFE_MAC2: SGMII via SerDes0 lane1

The S32G MACs are configured as follows:
+---------+--------------+------------------+
|                    | LANE 0               | LANE 1                        |
+---------+--------------+------------------+
| SERDES0    | GMAC (SGMII) | PFE_MAC2 (SGMII)  |
| SERDES1      | NOT USED        | PFE_MAC0 (SGMII) |
+---------+--------------+------------------+

Full U-Boot hwconfig:

hwconfig=pcie0:mode=sgmii,clock=ext,fmhz=100,xpcs_mode=both;pcie1:mode=sgmii,clock=ext,fmhz=100,xpcs_mode=0
pfeng_mode=enable,sgmii,rgmii,sgmii


DTS for port@2 (switch side):

			port@2 {
				reg = <2>;
				label = "OBC-1";
				ethernet = <&pfe_netif1>;
				phy-mode = "rgmii";
				rx-internal-delay-ps = <0>;
				tx-internal-delay-ps = <0>;
				fixed-link {
					speed = <1000>;
					full-duplex;
				};
			};


DTS for pfe_netif1 (MAC side):

&pfe_netif1 {
    phy-mode = "rgmii";
    status = "okay";
	fixed-link {
        speed = <1000>;
        full-duplex;
    };
};

 

PFE_MAC1 (pfe1) link state — confirmed up and correctly configured at the Linux/driver level:

dmesg at boot:

[    5.264108] pfeng 46000000.pfe: netif name: pfe1
[    5.274127] pfeng 46000000.pfe: netif(pfe1) linked phyif: 1
[    5.279692] pfeng 46000000.pfe: netif(pfe1) mode: std
[    5.284853] pfeng 46000000.pfe: netif(pfe1) HIFs: count 1 map 02
[    6.012884] pfeng 46000000.pfe pfe1 (uninitialized): Subscribe to HIF1
[    6.019438] pfeng 46000000.pfe pfe1 (uninitialized): Host LLTX disabled
[    6.026270] pfeng 46000000.pfe pfe1 (uninitialized): Enable HIF1
[    6.032374] pfeng 46000000.pfe pfe1 (uninitialized): setting MAC addr: 00:04:9f:be:ef:01
[    6.040545] pfeng 46000000.pfe pfe1 (uninitialized): PTP HW addend 0x80000000, max_adj configured to 46566128 ppb
[    6.060939] pfeng 46000000.pfe pfe1 (uninitialized): Registered PTP HW clock successfully on EMAC1
[    6.070441] pfeng 46000000.pfe pfe1: registered
[    6.207482] pfeng 46000000.pfe pfe1: configuring for fixed/rgmii link mode
[    6.214306] pfeng 46000000.pfe pfe1: Set TX clock to 125000000Hz
[    6.220158] pfeng 46000000.pfe pfe1: Link is Up - 1Gbps/Full - flow control off

[    5.257995] pfeng 46000000.pfe: EMAC0 interface mode: 4
[    5.290707] pfeng 46000000.pfe: EMAC1 interface mode: 9
[    5.323320] pfeng 46000000.pfe: EMAC2 interface mode: 4
[    5.354571] pfeng 46000000.pfe: Interface selected: EMAC0: 0x4 EMAC1: 0x9 EMAC2: 0x4
[    5.382609] pfeng 46000000.pfe: TX clock on EMAC0 for interface sgmii installed
[    5.390050] pfeng 46000000.pfe: RX clock on EMAC0 for interface sgmii installed
[    5.404998] pfeng 46000000.pfe: TX clock on EMAC1 for interface rgmii installed
[    5.419918] pfeng 46000000.pfe: Defer enabling of RX clock on EMAC1 for interface rgmii (ret: -5)
[    5.434235] pfeng 46000000.pfe: TX clock on EMAC2 for interface sgmii installed
[    5.448374] pfeng 46000000.pfe: RX clock on EMAC2 for interface sgmii installed
[    5.667058] pfeng 46000000.pfe: EMAC timestamp external mode bitmap: 0
[    5.998447] pfeng 46000000.pfe pfe0 (uninitialized): Registered PTP HW clock successfully on EMAC0
[    6.060939] pfeng 46000000.pfe pfe1 (uninitialized): Registered PTP HW clock successfully on EMAC1
[    6.130296] pfeng 46000000.pfe pfe2 (uninitialized): Registered PTP HW clock successfully on EMAC2
[    6.215040] pfeng 46000000.pfe: RX clock on EMAC1 for interface rgmii installed

 

Live DTB confirms the kernel matches the source DTS:

# cat /proc/device-tree/soc/pfe@46000000/ethernet@11/phy-mode
rgmii

ip a output:

6: pfe1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1536 qdisc mq state UP group default qlen 1000
link/ether 00:04:9f:be:ef:01 brd ff:ff:ff:ff:ff:ff
inet6 fe80::204:9fff:febe:ef01/64 scope link


All SJA1110 DSA slave ports correctly enumerated. This confirms the sja1105 DSA driver bound successfully
to pfe1 as the CPU port/DSA master and parsed the static config without error.

Clock tree: both TX and RX RGMII clocks enabled and attached to the correct consumer:

# cat /sys/kernel/debug/clk/clk_summary | grep pfe1
 pfe1_tx_mii                         0       0        0        125000000   0          0     50000      Y   deviceless                      no_connection_id         
 pfe1_rx_mii                         0       0        0        125000000   0          0     50000      Y   deviceless                      no_connection_id         
 pfe1_tx_rmii                        0       0        0        125000000   0          0     50000      Y   deviceless                      no_connection_id         
 pfe1_rx_rmii                        0       0        0        125000000   0          0     50000      Y   deviceless                      no_connection_id         
 pfe1_tx_rgmii                       1       1        0        125000000   0          0     50000      Y   ethernet@11                     tx_rgmii                 
 pfe1_rx_rgmii                       1       1        0        125000000   0          0     50000      Y   ethernet@11                     rx_rgmii                 
 pfe1_tx_sgmii                       0       0        0        125000000   0          0     50000      Y   deviceless                      no_connection_id         
 pfe1_rx_sgmii                       0       0        0        125000000   0          0     50000      Y   deviceless                      no_connection_id

 

So pfe1 is UP, LOWER_UP, correctly bound to the SJA1110 as DSA master, running in RGMII mode
with both clocks enabled — this rules out pfe1 being down, unbound, or misconfigured at the
Linux/driver level. The open question is specifically whether frames actually cross the
physical RGMII pins between PFE_MAC1 and SJA1110 port 2.

Issue:
No traffic appears to cross the RGMII bus between PFE_MAC1 and SJA1110 port 2 in either direction, despite everything on both sides of that bus being independently up:

Test 1 — S32G -> switch direction

Setup:

ip addr add 192.168.1.100/24 dev EPS-100bt1-9
ethtool -S pfe1 | grep '^ p02_' > before
tcpdump -i pfe1 -e -nn -c 20 > capture.txt &
arping -c 10 -I EPS-100bt1-9 192.168.1.6
ethtool -S pfe1 | grep '^ p02_' > after

 

# arping -c 10 -I EPS-100bt1-9 192.168.1.6
ARPING 192.168.1.6 from 192.168.1.100 EPS-100bt1-9
Sent 10 probes (10 broadcast(s))
Received 0 response(s)

 

$ cat capture.txt
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on pfe1, link-type NULL (BSD loopback), snapshot length 262144 bytes
18:08:36.352535 AF Unknown (4294967295), length 64:
        0x0000:  ffff 0004 9fbe ef01 dadb 0c09 0806 0001  ................
        0x0010:  0800 0604 0001 0004 9fbe ef01 c0a8 0164  ...............d
        0x0020:  ffff ffff ffff c0a8 0106 0000 0000 0000  ................
        0x0030:  0000 0000 0000 0000 0000 0000            ............
[... 9 more identical ARP frames, all correctly DSA-tagged (dadb 0c09) and well-formed,
     plus one unrelated IPv6 background frame interleaved ...]

 

# diff before after
--- before
+++ after
@@ -1,4 +1,4 @@
-     p02_: 1
+     p02_: 0
      p02_n_runt: 0
      p02_n_soferr: 0
      p02_n_alignerr: 0

 

# grep n_rxfrm before after
before:     p02_n_rxfrm: 0
after:      p02_n_rxfrm: 0


Test 2 — switch -> S32G direction
Setup Partner board is a separate SJA1105 switch based board.

# ping -c 10 -I t1-6 192.168.1.100 (run on a separate SJA1105/1110-family switch board
connected to our port 9 / 100BASE-T1 / EPS-100bt1-9)

 

# diff before after (ethtool -S EPS-100bt1-9)
- n_rxfrm: 0
+ n_rxfrm: 9 <- port 9 physically received 9 frames from the wire

 

 # diff before after 
- p02_n_txfrm: 0
+ p02_n_txfrm: 9 <- switch fabric forwarded all 9 toward the CPU port

 

# tcpdump -i pfe1 -e -nn -c 20 (same window)
listening on pfe1, link-type NULL (BSD loopback), snapshot length 262144 bytes
[-- nothing captured --]


Port 9 received 9 real frames; the fabric forwarded all 9 toward port 2 — but
nothing arrived at pfe1.

So the SJA1110's own fabric counters show all 9 frames successfully forwarded from port 9 to port 2's egress.

But tcpdump -i pfe1 -e -nn on the S32G during this exact test shows NOTHING received.

So the DSA/software layer on the S32G side believes it's sending (case 1). The switch's internal fabric believes it's sending toward the CPU port (case 2). Neither side has any confirmation that the other actually received anything across the physical RGMII bus. Every layer adjacent to this bus works individually; the bus itself has no confirmed successful crossing in either direction.

What's been ruled out so far:
- pfeng_mode / hwconfig (xpcs_mode) — confirmed correct; EMAC1 mode is RGMII (0x9), not SGMII (it was previously misconfigured as SGMII due to xpcs_mode=both on SerDes1 forcing PFE_MAC1's XPCS into SGMII; corrected to xpcs_mode=0 since PFE_MAC0 alone only needs XPCS0)
- PFE_MAC1 TX/RX clock enablement — confirmed enabled at the correct rate (125MHz) in clk_summary
- DSA tagging and CPU port binding — confirmed working (port netdevs exist, frames get tagged with the correct destination port in the DSA header)
- SJA1110 internal fabric/forwarding — confirmed working between two other ports (9 and 2) using real external traffic
- BASE-T1 link partner — confirmed passing real frames into the switch (port 9 n_rxfrm increments from genuine wire traffic)

What hasn't been ruled out / open questions:
- Whether 1000 Mbps RGMII with zero internal delay on both MAC and switch sides (rx/tx-internal-delay-ps=0, plain "rgmii" not "rgmii-id") is compatible without delay added by board trace length — have not yet tried forcing the link down to 100 Mbps as a timing-margin test

1. Is rx/tx-internal-delay-ps=0 on both ends at 1000 Mbps RGMII expected to work, or does this combination typically require delay compensation unless the PCB explicitly accounts for it?
2. Am I missing any other configuration?

Happy to share full register dumps, if required. Appreciate any pointers before we probe the PE_02-13 bus with a logic analyzer (limited probe access due to board layout, so it is not so convenient currently.

Thanks.

标记 (3)
0 项奖励
回复
2 回复数

2 小时之前
9 次查看
db16122
Contributor IV

any schematics sharing from hardware side for RGMII bus between PFE_MAC1 and SJA1110 port 2 ?

0 项奖励
回复

2 小时之前
11 次查看
Joey_z
NXP Employee
NXP Employee

Hi，pcentauri92

Thank you for your detail information

According to my understanding, there seems to be a problem with the communication when using PFE_MAC1 RGMAII and Port 2 of SJA1110A on your development board. Is that correct?

The default configuration of S32G-VNP-RDB3 is that PFE_MAC0/1 operates in SGMII mode and is connected to SJA1110. On your development board, why did you consider using RGMII mode? It is recommended to modify the corresponding software configuration.

BR

Joey

 

0 项奖励
回复