topic Re: T2080 SGMII (SD1_c/d) network ping failed in QorIQ

T2080 SGMII (SD1_c/d) network ping failed

xijungan — Sat, 20 Apr 2019 10:30:10 GMT

I have a T2080 board customized base on t2080rdb-pcie. RCW field SRDS_PRTCL_S1/S2 are configured as 0x6C/0x29, so SD1_C/D are multiplexed as SG1/SG2 (SGMII MAC1 and MAC2 of Frame Manager). Both ports are connected to PHYs (ar8031 x2) and then routed to 2 RJ45 connectors.

SD1_REFCLK2 (PLL2 reference clock) are 100Mhz, this is measured and confirmed. RCW field 'SRDS_PLL_REF_C
LK_SEL_S1' are configured to '00', that mean both PLL1 and PLL2 of SD1 are 100MHz. So, supplied reference clock and the setting are matched.

OS running is Linux, dts file configuration are:

/////////////////////////////////////////////////////////////////////

fman0: fman@400000 {
fm1mac1: ethernet@e0000 {
phy-handle = <&sgmii_phy3>;
phy-connection-type = "sgmii";
};

fm1mac2: ethernet@e2000 {
phy-handle = <&sgmii_phy4>;
phy-connection-type = "sgmii";
};

...

mdio0: mdio@fc000 {

...
sgmii_phy3: ethernet-phy@3 {
interrupts = <3 1 0 0>;
reg = <0x3>;
};
sgmii_phy4: ethernet-phy@4 {
interrupts = <10 1 0 0>;
reg = <0x0>;
};

...

};

...

};

PHY address of one ar8031 is 0, another one is 3, PHY address and mode (SGMII to 1000BaseT) were confirmed by measure pin of ar8031.

In the Linux rootfs console, instructions:

$ ifconfig fm1-mac1 192.168.1.7

// succeed, port brings up

$ ethtool fm1-mac1

// When insert network cable, 1000M/FD and linkup detected; when remove network cable, linkdown detected.

$ ping 192.168.1.2

// The opposite PC's ip is 192.168.1.2, ping failed

$ ifconfig fm1-mac1

// there are several tx-packets (caused by ping), no rx-packets, no error.

Test with 'fm1-mac2' get same result.

So,

1) The RCW configuration (serdes mux, serdes reference clock) are confirmed to be ok;

2) The measured serdes reference clock is exactly same to what configured in RCW;

3) The address of SGMII PHY (ar8031 x2) are confirmed to be ok;

4) The mode of SGMII PHY (ar8031 x2) are confirmed to be ok;

5) PHYs can correctly detect the link parameters (up/down, speed, fd/hd) when cable inserted or removed.

But, the 2 sgmii ports are unable to ping.

Are there any limit about the reference clock of serdes (ex. it must be ready at the REST release)? Or are there anything we missed to check and confirm?

could someone help me to identify the trouble?

Great thanks!

Re: T2080 SGMII (SD1_c/d) network ping failed

ufedor — Mon, 22 Apr 2019 08:48:17 GMT

Please provide U-Boot log as a text for inspection.

You wrote:

> there are several tx-packets (caused by ping), no rx-packets, no error.

Please run a packet analyzer on the 192.168.1.2 and check these packets.

Re: T2080 SGMII (SD1_c/d) network ping failed

xijungan — Mon, 22 Apr 2019 13:57:55 GMT

I have no board right now by my side, so I just can upload the log captured several days ago.

The only difference made is that 'SRDS_PRTCL_S2' was changed from 15h to 29h.

This modification redefined the function of serdes2, its nothing about sg1&sg2 of serdes1.

///////////////////////////////////////////////////////////////

U-Boot 2016.09-ge26eb70-dirty (Mar 03 2019 - 02:02:33 -0800)
CPU0: T2080E, Version: 1.1, (0x85380011)
Core: e6500, Version: 2.0, (0x80400120)
Clock Configuration:
       CPU0:1799.820 MHz, CPU1:1799.820 MHz, CPU2:1799.820 MHz, CPU3:1799.820 MHz,
       CCB:799.920 MHz,
       DDR:933.310 MHz (1866.620 MT/s data rate) (Asynchronous), IFC:799.920 MHz
       FMAN1: 699.930 MHz
       QMAN: 399.960 MHz
       PME:   799.920 MHz
L1:    D-cache 32 KiB enabled
       I-cache 32 KiB enabled
Reset Configuration Word (RCW):
       00000000: 1807001b 15000000 00000000 00000000
       00000010: 6c150002 00000000 ec02e000 c1000000
       00000020: 00000000 00000000 00000000 00018000
       00000030: 00000000 00000000 00000000 00000004
Board: T2080RDB, Board rev: 0x00 CPLD ver: 0x00, boot from NOR vBank0
SERDES Reference Clocks:
SD1_CLK1=156.25MHZ, SD1_CLK2=100.00MHZ
SD2_CLK1=100.00MHZ, SD2_CLK2=100.00MHZ
I2C:   ready
SPI:   ready
DRAM: 0--0x1b8
Initializing....using RAW_TIMING
starting at step 1 (STEP_GET_SPD)
Filling dimm parameters from board specific file
Detected UDIMM Fixed DDR3L on board
2 GiB left unmapped
4 GiB (DDR3, 64-bit, CL=11, ECC on)
       DDR Chip-Select Interleaving Mode: CS0+CS1
VID: Could not find voltage regulator on I2C.
Warning: Adjusting core voltage failed.
Flash: 128 MiB
L2:    2 MiB enabled
Corenet Platform Cache: 512 KiB enabled
Using SERDES1 Protocol: 108 (0x6c)
Using SERDES2 Protocol: 21 (0x15)
SEC0: RNG instantiated
NAND: 512 MiB
MMC:   FSL_SDHC: 0
EEPROM: NXID v1
PCIe1: Root Complex, no link, regs @ 0xfe240000
PCIe1: Bus 00 - 00
PCIe2: Root Complex, no link, regs @ 0xfe250000
PCIe2: Bus 01 - 01
PCIe3: disabled
PCIe4: Root Complex, no link, regs @ 0xfe270000
PCIe4: Bus 02 - 02
In:    serial
Out:   serial
Err:   serial
Net:   SerDes1 protocol 0x6c is used on board JHCD2080 for SGMII
port0 interface=0x2
set port0 to SGMII
port1 interface=0x2
set port1 to SGMII
port2 interface=0x7
set port2 to RGMII
port3 interface=0x7
set port3 to RGMII
port4 interface=0xd
port5 interface=0xd
port6 interface=0xd
port7 interface=0xd
Fman1: Uploading microcode version 108.4.5
PHY reset timed out
PHY reset timed out
FM_TGEC_MDIO:0 is connected to FM1@TGEC1. Reconnecting to FM1@TGEC2
FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3 [PRIME], FM1@DTSEC4, FM1@TGEC1, FM1@TGEC2
Hit any key to stop autoboot: 3 2 1 0
FM1@DTSEC3 Waiting for PHY auto negotiation to complete......... TIMEOUT !
FM1@DTSEC3: Could not initialize
FM1@DTSEC4 Waiting for PHY auto negotiation to complete......... TIMEOUT !
FM1@DTSEC4: Could not initialize
Using FM1@TGEC1 device
TFTP from server 192.168.4.4; our IP address is 192.168.4.5
Filename 'ramdisk.uboot'.
Load address: 0x2000000
Loading: *
Abort
Using FM1@TGEC1 device
TFTP from server 192.168.4.4; our IP address is 192.168.4.5
Filename 'uImage'.
Load address: 0x1000000
Loading: *
Abort
Using FM1@TGEC1 device
TFTP from server 192.168.4.4; our IP address is 192.168.4.5
Filename 't2080rdb.dtb'.
Load address: 0x3fe3000
Loading: *
Abort
WARNING: adjusting available memory to 30000000
Wrong Image Format for bootm command
ERROR: can't get kernel image!
=>
=>

///////////////////////////////////////////////////////////////

After linux boot up, ping 192.168.1.2 failed but have some tx-packets. In the ping duration, Wireshark on 192.168.1.2 captured nothing.

Re: T2080 SGMII (SD1_c/d) network ping failed

xijungan — Mon, 22 Apr 2019 14:12:19 GMT

Another try was made(Set Serdes1 PLL2 to 125M, replace the original 100M):
1. In RCW, configure Serdes1 PLL2 125MHz.

2. In u-boot, configure registers of CDCM6208 to output 'Serdes1 PLL2' reference clock 125MHz. Measured and confirmed it really changed to 125MHz.

But we got same result.

In u-boot print info, 'ORIG' means the original reg values of 6208, 'WRITE' means reg values to be write, 'AFTER' means reg values readback after the write.

Re: T2080 SGMII (SD1_c/d) network ping failed

ufedor — Tue, 23 Apr 2019 14:46:22 GMT

What are SYSCLK and DDRCLK frequencies?

What are frequencies of ALL applied SerDes reference clocks?

Re: T2080 SGMII (SD1_c/d) network ping failed

xijungan — Tue, 23 Apr 2019 15:39:11 GMT

SYSCLK is 66.66Mhz, DDRCLK is 133.33MHz.
SD1_REFCLK1 is 156.25Mhz, SD1_REFCLK2 is 100MHz (we also tried with 125Mhz).

SD2_REFCLK1 is 156.25Mhz, SD2_REFCLK2 is 100MHz.

Linux can boot up. Ramdisk rootfs works smoothly, without any panic errors.

Re: T2080 SGMII (SD1_c/d) network ping failed

xijungan — Tue, 23 Apr 2019 15:41:26 GMT

Are there any limitation to the powerup sequence (or clock sequence). For example, SD1_REFCLK1 should be stable at 100Mhz, just before the release of RESET.

Re: T2080 SGMII (SD1_c/d) network ping failed

ufedor — Tue, 23 Apr 2019 16:54:50 GMT

> SD1_REFCLK1 should be stable at 100Mhz, just before the release of RESET.

Yes, this is a must.

Re: T2080 SGMII (SD1_c/d) network ping failed

xijungan — Tue, 23 Apr 2019 23:14:37 GMT

Which part of the RM explained this? Can we give the serdes a reset after SD1_REFCLK1 changed and became stable?

~~-Original~~-

Re: T2080 SGMII (SD1_c/d) network ping failed

ufedor — Wed, 24 Apr 2019 02:36:59 GMT

> Which part of the RM explained this?

QorIQ T2080 Reference Manual, 4.6.1 Power-on reset sequence, Note.

The PLL reset is described in the 19.6.4.3 PLL Reset and Reconfiguration.