Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction Board: Custom S32G399A based module, derived from S32G-VNP-RDB3. PFE_MAC1 connected via RGMII (PE_02–PE_13) to an NXP SJA1110A switch port 2, configured as the DSA CPU port (in-tree sja1105 driver, kernel 6.x BSP43.0). Topology: - PFE_MAC0: SGMII via SerDes1 lane1, Mode 1 - PFE_MAC1: RGMII to SJA1110A port 2 (DSA CPU port) — the port in question - PFE_MAC2: SGMII via SerDes0 lane1 The S32G MACs are configured as follows: +---------+--------------+------------------+ | | LANE 0 | LANE 1 | +---------+--------------+------------------+ | SERDES0 | GMAC (SGMII) | PFE_MAC2 (SGMII) | | SERDES1 | NOT USED | PFE_MAC0 (SGMII) | +---------+--------------+------------------+ Full U-Boot hwconfig: hwconfig=pcie0:mode=sgmii,clock=ext,fmhz=100,xpcs_mode=both;pcie1:mode=sgmii,clock=ext,fmhz=100,xpcs_mode=0
pfeng_mode=enable,sgmii,rgmii,sgmii DTS for port@2 (switch side): port@2 {
reg = <2>;
label = "OBC-1";
ethernet = <&pfe_netif1>;
phy-mode = "rgmii";
rx-internal-delay-ps = <0>;
tx-internal-delay-ps = <0>;
fixed-link {
speed = <1000>;
full-duplex;
};
}; DTS for pfe_netif1 (MAC side): &pfe_netif1 {
phy-mode = "rgmii";
status = "okay";
fixed-link {
speed = <1000>;
full-duplex;
};
}; PFE_MAC1 (pfe1) link state — confirmed up and correctly configured at the Linux/driver level: dmesg at boot: [ 5.264108] pfeng 46000000.pfe: netif name: pfe1
[ 5.274127] pfeng 46000000.pfe: netif(pfe1) linked phyif: 1
[ 5.279692] pfeng 46000000.pfe: netif(pfe1) mode: std
[ 5.284853] pfeng 46000000.pfe: netif(pfe1) HIFs: count 1 map 02
[ 6.012884] pfeng 46000000.pfe pfe1 (uninitialized): Subscribe to HIF1
[ 6.019438] pfeng 46000000.pfe pfe1 (uninitialized): Host LLTX disabled
[ 6.026270] pfeng 46000000.pfe pfe1 (uninitialized): Enable HIF1
[ 6.032374] pfeng 46000000.pfe pfe1 (uninitialized): setting MAC addr: 00:04:9f:be:ef:01
[ 6.040545] pfeng 46000000.pfe pfe1 (uninitialized): PTP HW addend 0x80000000, max_adj configured to 46566128 ppb
[ 6.060939] pfeng 46000000.pfe pfe1 (uninitialized): Registered PTP HW clock successfully on EMAC1
[ 6.070441] pfeng 46000000.pfe pfe1: registered
[ 6.207482] pfeng 46000000.pfe pfe1: configuring for fixed/rgmii link mode
[ 6.214306] pfeng 46000000.pfe pfe1: Set TX clock to 125000000Hz
[ 6.220158] pfeng 46000000.pfe pfe1: Link is Up - 1Gbps/Full - flow control off
[ 5.257995] pfeng 46000000.pfe: EMAC0 interface mode: 4
[ 5.290707] pfeng 46000000.pfe: EMAC1 interface mode: 9
[ 5.323320] pfeng 46000000.pfe: EMAC2 interface mode: 4
[ 5.354571] pfeng 46000000.pfe: Interface selected: EMAC0: 0x4 EMAC1: 0x9 EMAC2: 0x4
[ 5.382609] pfeng 46000000.pfe: TX clock on EMAC0 for interface sgmii installed
[ 5.390050] pfeng 46000000.pfe: RX clock on EMAC0 for interface sgmii installed
[ 5.404998] pfeng 46000000.pfe: TX clock on EMAC1 for interface rgmii installed
[ 5.419918] pfeng 46000000.pfe: Defer enabling of RX clock on EMAC1 for interface rgmii (ret: -5)
[ 5.434235] pfeng 46000000.pfe: TX clock on EMAC2 for interface sgmii installed
[ 5.448374] pfeng 46000000.pfe: RX clock on EMAC2 for interface sgmii installed
[ 5.667058] pfeng 46000000.pfe: EMAC timestamp external mode bitmap: 0
[ 5.998447] pfeng 46000000.pfe pfe0 (uninitialized): Registered PTP HW clock successfully on EMAC0
[ 6.060939] pfeng 46000000.pfe pfe1 (uninitialized): Registered PTP HW clock successfully on EMAC1
[ 6.130296] pfeng 46000000.pfe pfe2 (uninitialized): Registered PTP HW clock successfully on EMAC2
[ 6.215040] pfeng 46000000.pfe: RX clock on EMAC1 for interface rgmii installed Live DTB confirms the kernel matches the source DTS: # cat /proc/device-tree/soc/pfe@46000000/ethernet@11/phy-mode
rgmii ip a output: 6: pfe1: mtu 1536 qdisc mq state UP group default qlen 1000
link/ether 00:04:9f:be:ef:01 brd ff:ff:ff:ff:ff:ff
inet6 fe80::204:9fff:febe:ef01/64 scope link All SJA1110 DSA slave ports correctly enumerated. This confirms the sja1105 DSA driver bound successfully to pfe1 as the CPU port/DSA master and parsed the static config without error. Clock tree: both TX and RX RGMII clocks enabled and attached to the correct consumer: # cat /sys/kernel/debug/clk/clk_summary | grep pfe1
pfe1_tx_mii 0 0 0 125000000 0 0 50000 Y deviceless no_connection_id
pfe1_rx_mii 0 0 0 125000000 0 0 50000 Y deviceless no_connection_id
pfe1_tx_rmii 0 0 0 125000000 0 0 50000 Y deviceless no_connection_id
pfe1_rx_rmii 0 0 0 125000000 0 0 50000 Y deviceless no_connection_id
pfe1_tx_rgmii 1 1 0 125000000 0 0 50000 Y ethernet@11 tx_rgmii
pfe1_rx_rgmii 1 1 0 125000000 0 0 50000 Y ethernet@11 rx_rgmii
pfe1_tx_sgmii 0 0 0 125000000 0 0 50000 Y deviceless no_connection_id
pfe1_rx_sgmii 0 0 0 125000000 0 0 50000 Y deviceless no_connection_id So pfe1 is UP, LOWER_UP, correctly bound to the SJA1110 as DSA master, running in RGMII mode with both clocks enabled — this rules out pfe1 being down, unbound, or misconfigured at the Linux/driver level. The open question is specifically whether frames actually cross the physical RGMII pins between PFE_MAC1 and SJA1110 port 2. Issue: No traffic appears to cross the RGMII bus between PFE_MAC1 and SJA1110 port 2 in either direction, despite everything on both sides of that bus being independently up: Test 1 — S32G -> switch direction Setup: ip addr add 192.168.1.100/24 dev EPS-100bt1-9
ethtool -S pfe1 | grep '^ p02_' > before
tcpdump -i pfe1 -e -nn -c 20 > capture.txt &
arping -c 10 -I EPS-100bt1-9 192.168.1.6
ethtool -S pfe1 | grep '^ p02_' > after # arping -c 10 -I EPS-100bt1-9 192.168.1.6
ARPING 192.168.1.6 from 192.168.1.100 EPS-100bt1-9
Sent 10 probes (10 broadcast(s))
Received 0 response(s) $ cat capture.txt
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on pfe1, link-type NULL (BSD loopback), snapshot length 262144 bytes
18:08:36.352535 AF Unknown (4294967295), length 64:
0x0000: ffff 0004 9fbe ef01 dadb 0c09 0806 0001 ................
0x0010: 0800 0604 0001 0004 9fbe ef01 c0a8 0164 ...............d
0x0020: ffff ffff ffff c0a8 0106 0000 0000 0000 ................
0x0030: 0000 0000 0000 0000 0000 0000 ............
[... 9 more identical ARP frames, all correctly DSA-tagged (dadb 0c09) and well-formed,
plus one unrelated IPv6 background frame interleaved ...] # diff before after
--- before
+++ after
@@ -1,4 +1,4 @@
- p02_: 1
+ p02_: 0
p02_n_runt: 0
p02_n_soferr: 0
p02_n_alignerr: 0 # grep n_rxfrm before after
before: p02_n_rxfrm: 0
after: p02_n_rxfrm: 0 Test 2 — switch -> S32G direction Setup Partner board is a separate SJA1105 switch based board. # ping -c 10 -I t1-6 192.168.1.100 (run on a separate SJA1105/1110-family switch board
connected to our port 9 / 100BASE-T1 / EPS-100bt1-9) # diff before after (ethtool -S EPS-100bt1-9)
- n_rxfrm: 0
+ n_rxfrm: 9 <- port 9 physically received 9 frames from the wire # diff before after
- p02_n_txfrm: 0
+ p02_n_txfrm: 9 <- switch fabric forwarded all 9 toward the CPU port # tcpdump -i pfe1 -e -nn -c 20 (same window)
listening on pfe1, link-type NULL (BSD loopback), snapshot length 262144 bytes
[-- nothing captured --] Port 9 received 9 real frames; the fabric forwarded all 9 toward port 2 — but nothing arrived at pfe1. So the SJA1110's own fabric counters show all 9 frames successfully forwarded from port 9 to port 2's egress. But tcpdump -i pfe1 -e -nn on the S32G during this exact test shows NOTHING received. So the DSA/software layer on the S32G side believes it's sending (case 1). The switch's internal fabric believes it's sending toward the CPU port (case 2). Neither side has any confirmation that the other actually received anything across the physical RGMII bus. Every layer adjacent to this bus works individually; the bus itself has no confirmed successful crossing in either direction. What's been ruled out so far: - pfeng_mode / hwconfig (xpcs_mode) — confirmed correct; EMAC1 mode is RGMII (0x9), not SGMII (it was previously misconfigured as SGMII due to xpcs_mode=both on SerDes1 forcing PFE_MAC1's XPCS into SGMII; corrected to xpcs_mode=0 since PFE_MAC0 alone only needs XPCS0) - PFE_MAC1 TX/RX clock enablement — confirmed enabled at the correct rate (125MHz) in clk_summary - DSA tagging and CPU port binding — confirmed working (port netdevs exist, frames get tagged with the correct destination port in the DSA header) - SJA1110 internal fabric/forwarding — confirmed working between two other ports (9 and 2) using real external traffic - BASE-T1 link partner — confirmed passing real frames into the switch (port 9 n_rxfrm increments from genuine wire traffic) What hasn't been ruled out / open questions: - Whether 1000 Mbps RGMII with zero internal delay on both MAC and switch sides (rx/tx-internal-delay-ps=0, plain "rgmii" not "rgmii-id") is compatible without delay added by board trace length — have not yet tried forcing the link down to 100 Mbps as a timing-margin test 1. Is rx/tx-internal-delay-ps=0 on both ends at 1000 Mbps RGMII expected to work, or does this combination typically require delay compensation unless the PCB explicitly accounts for it? 2. Am I missing any other configuration? Happy to share full register dumps, if required. Appreciate any pointers before we probe the PE_02-13 bus with a logic analyzer (limited probe access due to board layout, so it is not so convenient currently. Thanks. Re: Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction Hi @Joey_z @db16122 I am attaching the relevant sections of the schematics here. The connection flow is as follows: We use PE_02 to PE_13 on the S32G3 chip shown in s32g3_pfe_mac1_connections.png for the PFE_MAC1. They go to a board to board connector (shown in Board_to_board_connector.png) that routes these signals to a different board that has the switch. The switch connections are shown in SJA1110_A.png and SJA1110_B.png. So the connection is PE_xx pins -> board connectors -> switch (SJA1110) Please let me know if you have any questions. I have also raised a support ticket ( #00990408) with the same details. Re: Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction Hi,pcentauri92
Thank you for your reply.
Please provide me with the schematic diagrams related to your ETH, particularly the ones for PFE_MCA1 and SJA1110 sections. You can create an internal support system case. In the information description, @Joey, then provide your schematic diagram information. Refer to this website: https://support.nxp.com
BR
Joey Re: Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction Hi @Joey_z , Thank you for the response. The module in question here is a custom design that uses the S32G399A chip along with the NXP SJA1110A ethernet switch. We based this design on the S32G-VNP-RDB3 development platform but we made quite a few changes from the base design. The PFE_MAC1 using RGMII is one of those changes. I am also attaching the dts file override where we change the PFE_MAC1 mode and pinmux here. PFE_MAC1 mode configuration: /* pfe_mdio1 is already disabled in the base config in s32gxxxa-rdb.dtsi */
&pfe_mdio1 {
/* occupied by GMAC0 */
status = "disabled";
};
/*
* pfe_netif1 = PFE_MAC1 — management port to Switch-A port 2.
* Overrides the base "sgmii" stub in s32gxxxa-rdb.dtsi.
* Plain "rgmii" (no -id/-txid) since both MAC and switch add zero delay.
* No phy-handle: the link partner is the SJA1110A switch, described as a
* fixed-link on switch port@2. MDIO is not needed for link management here.
*/
&pfe_netif1 {
phy-mode = "rgmii";
status = "okay";
fixed-link {
speed = <1000>;
full-duplex;
};
}; PFE_MAC1 pinmux: /*
* PFE_MAC1 RGMII pinmux — management port to Switch-A.
*
* All RX pad SSS values confirmed from S32G3 IOMUX spreadsheet.
* TX path: output pads only, no IMCR needed.
* RX path: input pads + IMCR registers to route pads into PFE_MAC1.
*
* Note: PE_07 (TXD3) uses FUNC3, not FUNC2. Similarly PE_08 (RX_CLK) output uses FUNC3; its IMCR (CR#859) uses FUNC2.
*/
pfe1rgmii_pins: pfe1rgmii_pins {
/* TX outputs: PE_02=TX_CLK, PE_03=TX_EN, PE_04=TXD0,
PE_05=TXD1, PE_06=TXD2 PE_07 (TXD3) */
pfe1rgmii_grp0 {
pinmux = , /* PE_02: PFE_MAC1_TX_CLK */
, /* PE_03: PFE_MAC1_TX_EN */
, /* PE_04: PFE_MAC1_TXD0 */
, /* PE_05: PFE_MAC1_TXD1 */
, /* PE_06: PFE_MAC1_TXD2 */
; /* PE_07: PFE_MAC1_TXD3 */
output-enable;
slew-rate = ;
};
/* RX inputs — pads set to FUNC0 (input mode); routing into PFE_MAC1
is handled by the IMCR entries in pfe1rgmii_grp2 below.
NXP input mux pattern: pad=FUNC0 + IMCR=FUNC2 */
pfe1rgmii_grp1 {
pinmux = , /* PE_08: input */
, /* PE_09: input */
, /* PE_10: input */
, /* PE_11: input */
, /* PE_12: input */
; /* PE_13: input */
input-enable;
slew-rate = ;
};
/* IMCR input mux — selects which pad drives each PFE_MAC1 RX signal.
CR#866 routes PE_02 (TX_CLK pad) back into PFE_MAC1_TX_CLK_I; required
even for RGMII TX because the MAC samples its own TX_CLK internally.
All entries at FUNC2 per S32G3 IOMUX spreadsheet. */
pfe1rgmii_grp2 {
pinmux = , /* CR#866: PFE_MAC1_TX_CLK_I ← PE_02 */
, /* CR#859: PFE_MAC1_RX_CLK_I ← PE_08 */
, /* CR#865: PFE_MAC1_RXDV_I ← PE_09 */
, /* CR#861: PFE_MAC1_RXD_I[0] ← PE_10 */
, /* CR#862: PFE_MAC1_RXD_I[1] ← PE_11 */
, /* CR#863: PFE_MAC1_RXD_I[2] ← PE_12 */
; /* CR#864: PFE_MAC1_RXD_I[3] ← PE_13 */
};
}; Please let me know if you need any other information. Re: Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction any schematics sharing from hardware side for RGMII bus between PFE_MAC1 and SJA1110 port 2 ? Re: Custom S32G399A board: No frames cross switch-facing RGMII bus in either direction Hi，pcentauri92
Thank you for your detail information
According to my understanding, there seems to be a problem with the communication when using PFE_MAC1 RGMAII and Port 2 of SJA1110A on your development board. Is that correct?
The default configuration of S32G-VNP-RDB3 is that PFE_MAC0/1 operates in SGMII mode and is connected to SJA1110. On your development board, why did you consider using RGMII mode? It is recommended to modify the corresponding software configuration.
BR
Joey
查看全文
