T1042 RGMII EC2 not working with SerDes option 0x8E

stefanlange · ‎05-12-2016

Hello NXP team,

we have a design with a custom NXP T1042 PowerPC Module located on a custom baseboard.

Both module and baseboard are new designs.

I here have a support request regarding the SerDes options, especially the behaviour of the RGMII interfaces in some configurations.

Our design is implemented in such a way that multiple SerDes options can be used.

T1040 reference manual shows Figure 31-1 Supported SerDesOptions SRDS_PRTCL_S1_RCW.

Two of the SRDS_PRTCL_S1_RCW options we use are 0x86 and 0x8E.

Setting the RCW to option 0x86, all SerDes and both RGMII interfaces work as expected.

Setting the RCW to option 0x8E, all SerDes interfaces work as expected. However the FMAN MAC5 EC2 RGMII does not work anymore.

(Neither does the EC1, but this is understood as indicated in Figure 31-1)

In my understanding of Figure 31-1, EC2 RGMII should work in configuration 0x8E.

Following dump shows the full RCW that we have configured:

Reset Configuration Word (RCW):

00000000: 0c10000e 0e000000 00000000 00000000

00000010: 8e000000 00000001 ec027000 01000000

00000020: 00000000 00000000 00000000 0002a800

00000030: 00000000 11fe5005 00000000 00000000

Have we overlooked something or done anything wrong with the RCW?

As the interface works correctly in a different RCW, I am confident that the malfunction is not due to the hardware.

Following dump shows the register values of the EMAC5:

=> md 0xfe4e8300

(if connected to Gigabit Switch): (emac5)

fe4e8300: 00005006 0000d000 00000000 00000000 ..P.............

GB, RGMII mode, GMII mode

1 - a valid link is established by the RGMII PHY (if it supports the optional in-band signaling)

10 - 1 Gbps (either controlled by in-band RGMII PHY status or by forced speed settings)

1 - RGMII full duplex link is established (bit is valid if PHY supports the optional in-band signaling)

(if connected to 100MB Switch):

fe4e8300: 00001002 0000b000 00000000 00000000

100MBit, non RGMII mode, GMII mode?

1 - a valid link is established by the RGMII PHY (if it supports the optional in-band signaling)

01 - 100 Mbps (either controlled by in-band RGMII PHY status or by forced speed settings)

1 - RGMII full duplex link is established (bit is valid if PHY supports the optional in-band signaling)

Register values are a little bit irritating, showing 100MBit: non RGMII mode, GMII mode?

Apart from this, register values seem ok.

Also the phy's MAC/RGMII-relevant register values look correct. An MDI link is established.

Has anybody ever tried SerDes configuration 0x8E?

It was not included in the Uboot source files (both denx git and SDK), so I added this configuration manually to arch/powerpc/cpu/mpc85xx/t1040_serdes.c

static u8 serdes_cfg_tbl[][SRDS_MAX_LANES] = {

..

[0x8D] = {PCIE1, SGMII_SW1_MAC3, SGMII_SW1_MAC1, SGMII_SW1_MAC2,

PCIE2, SGMII_SW1_MAC6, SGMII_SW1_MAC4, SGMII_SW1_MAC5},

[0x8E] = {PCIE1, SGMII_FM1_DTSEC3, SGMII_FM1_DTSEC1, SGMII_FM1_DTSEC2,

AURORA, PCIE3, SGMII_FM1_DTSEC4, SATA1},

[0x8F] = {PCIE1, SGMII_FM1_DTSEC3, SGMII_FM1_DTSEC1, SGMII_FM1_DTSEC2,

AURORA, NONE, SGMII_FM1_DTSEC4, SGMII_FM1_DTSEC5},

..

Is there anything else I can look at? Maybe you can shed some light.

Thanks and best regards,

Stefan

bootlog

########

U-Boot 2015.07-00014-gdf0d8ea-dirty (May 12 2016 - 10:49:59 +0200)

CPU0: T1042, Version: 1.1, (0x85200211)

Core: e5500, Version: 2.1, (0x80241021)

Clock Configuration:

CPU0:1400 MHz, CPU1:1400 MHz, CPU2:1400 MHz, CPU3:1400 MHz,

CCB:600 MHz,

DDR:800 MHz (1600 MT/s data rate) (Asynchronous), IFC:150 MHz

QE:300 MHz

FMAN1: 600 MHz

QMAN: 300 MHz

PME: 300 MHz

L1: D-cache 32 KiB enabled

I-cache 32 KiB enabled

Reset Configuration Word (RCW):

00000000: 0c10000e 0e000000 00000000 00000000

00000010: 8e000000 00000001 ec027000 01000000

00000020: 00000000 00000000 00000000 0002a800

00000030: 00000000 11fe5005 00000000 00000000

Board: TQMT1042

I2C: ready

DRAM: Initializing....using SPD

Detected UDIMM Fixed DDR on board

2 GiB (DDR3, 64-bit, CL=11, ECC on)

Flash: 128 MiB

L2: 256 KiB enabled

Corenet Platform Cache: 256 KiB enabled

Using SERDES1 Protocol: 142 (0x8e)

MMC: FSL_SDHC: 0

QE microcode not found

PCIe1: Root Complex, no link, regs @ 0xfe240000

PCIe1: Bus 00 - 00

PCIe2: disabled

PCIe3: Root Complex, x1 gen1, regs @ 0xfe260000

02:00.0 - 8086:10b9 - Network controller

PCIe3: Bus 01 - 02

PCIe4: disabled

In: serial

Out: serial

Err: serial

Net: Initializing Fman

Eth: configuring FM1_DTSEC1 as SGMII

Eth: configuring FM1_DTSEC2 as SGMII

Eth: configuring FM1_DTSEC3 as SGMII

Eth: configuring FM1_DTSEC4 as SGMII

Eth: configuring FM1_DTSEC5 as RGMII

Fman1: Uploading microcode version 107.4.2

Phy 3 not found

PHY reset timed out

e1000: 00:1b:21:33:5c:5d

FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5 [PRIME], e1000#0

Warning: e1000#0 MAC addresses don't match:

Address in SROM is 00:1b:21:33:5c:5d

Address in environment is 68:05:ca:37:92:88

Hit any key to stop autoboot: 0

=> ping $serverip

Using FM1@DTSEC5 device

ping failed; host 192.168.11.100 is not alive

=> mdio list

FSL_MDIO0:

1 - Generic PHY <--> FM1@DTSEC4

3 - Generic PHY <--> FM1@DTSEC2

5 - Generic PHY <--> FM1@DTSEC5

28 - Generic PHY <--> FM1@DTSEC3

29 - Generic PHY <--> FM1@DTSEC1

=> mii info

PHY 0x01: OUI = 0x80028, Model = 0x23, Rev = 0x01, 10baseT, HDX

PHY 0x05: OUI = 0x80028, Model = 0x23, Rev = 0x01, 100baseT, FDX

PHY 0x0E: OUI = 0x80028, Model = 0x23, Rev = 0x01, 10baseT, HDX

PHY 0x1C: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

PHY 0x1D: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

PHY 0x1E: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

PHY 0x1F: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

=>

ufedor · ‎05-28-2016

Please ensure that SCFG_EMIIOCR[GTXCLKSEL] =1 when 0x8E is selected.

View solution in original post

pranavmadhu · ‎05-23-2016

Hi,

Please have a look at the board file and the device tree for u-boot and verify that the address for the interfaces are provided correctly. I'm not sure that this will be the problem as it is working correct with the 0x86 RCW.

ufedor · ‎05-13-2016

Is the Erratum "A-008560: RGMII AC timing specifications for tSKRGT_TX and tSKRGT_RX are not met" addressed in your design?

stefanlange · ‎05-16-2016

We have implmented respective delay values within the phy's RGMII skew registers. Hence the erratum can be seen as addressed.

Besides, the interface is working 100% correct with the 0x86 RCW.

ufedor · ‎05-20-2016

fe4e8300: 00001002 0000b000 00000000 00000000 100MBit, non RGMII mode, GMII mode?

The IF_STATUS contents:

RGLINK = 1 - a valid link is established by the RGMII PHY (if it supports the optional in-band signaling)

RGSP = 01 - 100 Mbps (either controlled by in-band RGMII PHY status or by forced speed settings)

RGFD = 1 - RGMII full duplex link is established (bit is valid if PHY supports the optional in-band signaling) Register values are a little bit irritating, showing 100MBit: non RGMII mode

This is NOT controlled by RCW. It is SW programming only.

I think the issue may be caused by IF_MODE[RG]=0.

Please note that it is recommended to build all components from the same release - e.g., it is not correct to use u-boot from SDK v1.7 and Linux from SDK v1.9.

Please keep everything from the same version of the SDK.

stefanlange · ‎05-23-2016

Hello Ufedor,

I noted that the IF_MODE Register values are a little bit different after the first ping.

See below respective logs in both RCW configurations.

I do not think the issue is related to IF_MODE[RG]=0.

I unfortunately do not have a Freescale T1042 RDB at my site to counter-test this configuration with the reference hardware.

Do you have the chance to test RCW configuration 0x8E at your place?

Best regards,

Stefan

0x86:

#####

Using SERDES1 Protocol: 134 (0x86)

..

FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5 [PRIME]

Hit any key to stop autoboot: 0

=> md 0xfe4e8300

fe4e8300: 00001002 0000b000 00000000 00000000 ................

..

=> ping $serverip

Using FM1@DTSEC5 device

host 192.168.11.100 is alive

=> md 0xfe4e8300

fe4e8300: 00001006 0000b000 00000000 00000000 ................

0x8e:

#####

Using SERDES1 Protocol: 142 (0x8e)

..

FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5 [PRIME]

Hit any key to stop autoboot: 0

=> md 0xfe4e8300

fe4e8300: 00001002 0000b000 00000000 00000000 ................

=> ping $serverip

Using FM1@DTSEC5 device

ping failed; host 192.168.11.100 is not alive

=> md 0xfe4e8300

fe4e8300: 00001006 0000b000 00000000 00000000 ................

ufedor · ‎05-28-2016

Please ensure that SCFG_EMIIOCR[GTXCLKSEL] =1 when 0x8E is selected.

stefanlange · ‎06-07-2016

Hello ufedor,

thank you, this information was key.

With setting the register value and providing the 125 MHz GTX clock to EC2_GTX_CLK125 , the RGMII2 interface is now working in UBoot with SerDes config 0x8E.

Ping is successful. I will perform further tests soon.

Best regards,

Stefan

ufedor · ‎05-23-2016

I wrote:

"Please note that it is recommended to build all components from the same release - e.g., it is not correct to use u-boot from SDK v1.7 and Linux from SDK v1.9."

Please confirm that the latest tests were performed with U-Boot from SDK 1.9.

ufedor · ‎05-12-2016

Please provide similar data for the protocol 0x86 - i.e. dumps for the EMAC5 and U-Boot log.

stefanlange · ‎05-12-2016

Hello Ufedor,

see below the bootlog and dumps for the 0x86 configuration:

=> md 0xfe4e8300

fe4e8300: 00005006 0000d000 00000000 00000000 ..P.............

(GBit Switch attached)

U-Boot 2015.07-00018-g70ad811-dirty (May 12 2016 - 16:41:10 +0200)

CPU0: T1042, Version: 1.1, (0x85200211)

Core: e5500, Version: 2.1, (0x80241021)

Clock Configuration:

CPU0:1400 MHz, CPU1:1400 MHz, CPU2:1400 MHz, CPU3:1400 MHz,

CCB:600 MHz,

DDR:800 MHz (1600 MT/s data rate) (Asynchronous), IFC:150 MHz

QE:300 MHz

FMAN1: 600 MHz

QMAN: 300 MHz

PME: 300 MHz

L1: D-cache 32 KiB enabled

I-cache 32 KiB enabled

Reset Configuration Word (RCW):

00000000: 0c10000e 0e000000 00000000 00000000

00000010: 86000000 00000001 ec027000 01000000

00000020: 00000000 00000000 00000000 0002a800

00000030: 00000000 01fe5005 00000000 00000000

Board: TQMT1042

I2C: ready

DRAM: Initializing....using SPD

Detected UDIMM Fixed DDR on board

2 GiB (DDR3, 64-bit, CL=11, ECC on)

Flash: 128 MiB

L2: 256 KiB enabled

Corenet Platform Cache: 256 KiB enabled

Using SERDES1 Protocol: 134 (0x86)

MMC: FSL_SDHC: 0

QE microcode not found

PCIe1: Root Complex, no link, regs @ 0xfe240000

PCIe1: Bus 00 - 00

PCIe2: Root Complex, no link, regs @ 0xfe250000

PCIe2: Bus 01 - 01

PCIe3: Root Complex, x1 gen1, regs @ 0xfe260000

03:00.0 - 8086:10b9 - Network controller

PCIe3: Bus 02 - 03

PCIe4: Root Complex, no link, regs @ 0xfe270000

PCIe4: Bus 04 - 04

In: serial

Out: serial

Err: serial

Net: Initializing Fman

Eth: configuring FM1_DTSEC1 as SGMII

Eth: configuring FM1_DTSEC2 as SGMII

Eth: configuring FM1_DTSEC3 as SGMII

Eth: configuring FM1_DTSEC4 as RGMII

Eth: configuring FM1_DTSEC5 as RGMII

Fman1: Uploading microcode version 107.4.2

Phy 3 not found

PHY reset timed out

e1000: 00:1b:21:33:5c:5d

FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5 [PRIME], e1000#0

Warning: e1000#0 MAC addresses don't match:

Address in SROM is 00:1b:21:33:5c:5d

Address in environment is 68:05:ca:37:92:88

Hit any key to stop autoboot: 0

=>

=> ping $serverip

Using FM1@DTSEC5 device

host 192.168.11.100 is alive

=> mii info

PHY 0x01: OUI = 0x80028, Model = 0x23, Rev = 0x01, 10baseT, HDX

PHY 0x05: OUI = 0x80028, Model = 0x23, Rev = 0x01, 1000baseT, FDX

PHY 0x0E: OUI = 0x80028, Model = 0x23, Rev = 0x01, 10baseT, HDX

PHY 0x1C: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

PHY 0x1D: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

PHY 0x1E: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

PHY 0x1F: OUI = 0x5043, Model = 0x1C, Rev = 0x00, 10baseT, HDX

ufedor · ‎05-13-2016

It is not clear how SerDes configuration change could affect EC2 mEMAC5 operation.

Please provide complete raw memory dumps of the mEMAC5 registers for 0x86 and 0x8E.

stefanlange · ‎05-13-2016

Sorry, I should have provided those before.

Here it is, see below.

So far I see no differences. I even double-checked, not to accidentially induce a copy-paste error.

Additional note:

"Phy 3 not found

PHY reset timed out"

is caused by a not-populated ethernet phy on my prototype, which is however not related to the interface and issue we are discussing here.

Thank you and best regards,

Stefan

0x86

####

U-Boot 2015.07-00018-g70ad811-dirty (May 12 2016 - 16:41:10 +0200)

..

Reset Configuration Word (RCW):

00000000: 0c10000e 0e000000 00000000 00000000

00000010: 86000000 00000001 ec027000 01000000

00000020: 00000000 00000000 00000000 0002a800

00000030: 00000000 01fe5005 00000000 00000000

..

Net: Initializing Fman

Eth: configuring FM1_DTSEC1 as SGMII

Eth: configuring FM1_DTSEC2 as SGMII

Eth: configuring FM1_DTSEC3 as SGMII

Eth: configuring FM1_DTSEC4 as RGMII

Eth: configuring FM1_DTSEC5 as RGMII

Fman1: Uploading microcode version 107.4.2

Phy 3 not found

PHY reset timed out

e1000: 00:1b:21:33:5c:5d

FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3, FM1@DTSEC4, FM1@DTSEC5 [PRIME], e1000#0

Warning: e1000#0 MAC addresses don't match:

Address in SROM is 00:1b:21:33:5c:5d

Address in environment is 68:05:ca:37:92:88

Hit any key to stop autoboot: 0

=> md 0xfe4e8000

fe4e8000: 0001011e 00000000 00000840 00000000 ...........@....

fe4e8010: 00000000 00000800 00000000 00000020 ...............

fe4e8020: 00100020 00000000 00000000 00000000 ... ............

fe4e8030: 00001448 00000000 00000000 00000000 ...H............

fe4e8040: 00000060 0000000c 00000000 00000000 ...`............

fe4e8050: 00000000 00000000 00000000 00000000 ................

fe4e8060: 00000000 00000000 00000000 00000000 ................

fe4e8070: 00000000 00000000 00000000 00000000 ................

fe4e8080: 00000000 00000000 00000000 00000000 ................

fe4e8090: 00000000 00000000 00000000 00000000 ................

fe4e80a0: 00000000 00000000 00000000 00000000 ................

fe4e80b0: 00000000 00000000 00002000 00000000 .......... .....

fe4e80c0: 00000000 00000000 00000000 00000000 ................

fe4e80d0: 00000000 00000000 00000000 00000000 ................

fe4e80e0: 00000000 00000000 00000000 00000000 ................

fe4e80f0: 00000000 00000000 00000000 00000000 ................

=> md 0xfe4e8100

fe4e8100: 00000000 00000000 00000000 00000000 ................

fe4e8110: 00000000 00000000 00000000 00000000 ................

fe4e8120: 00000000 00000000 00000000 00000000 ................

fe4e8130: 00000000 00000000 00000000 00000000 ................

fe4e8140: 00000000 00000000 00000000 00000000 ................

fe4e8150: 00000000 00000000 00000000 00000000 ................

fe4e8160: 00000000 00000000 00000000 00000000 ................

fe4e8170: 00000000 00000000 00000000 00000000 ................

fe4e8180: 00000000 00000000 00000000 00000000 ................

fe4e8190: 00000000 00000000 00000000 00000000 ................

fe4e81a0: 00000000 00000000 00000000 00000000 ................

fe4e81b0: 00000000 00000000 00000000 00000000 ................

fe4e81c0: 00000000 00000000 00000000 00000000 ................

fe4e81d0: 00000000 00000000 00000000 00000000 ................

fe4e81e0: 00000000 00000000 00000000 00000000 ................

fe4e81f0: 00000000 00000000 00000000 00000000 ................

=> md 0xfe4e8200

fe4e8200: 00000000 00000000 00000000 00000000 ................

fe4e8210: 00000000 00000000 00000000 00000000 ................

fe4e8220: 00000000 00000000 00000000 00000000 ................

fe4e8230: 00000000 00000000 00000000 00000000 ................

fe4e8240: 00000000 00000000 00000000 00000000 ................

fe4e8250: 00000000 00000000 00000000 00000000 ................

fe4e8260: 00000000 00000000 00000000 00000000 ................

fe4e8270: 00000000 00000000 00000000 00000000 ................

fe4e8280: 00000000 00000000 00000000 00000000 ................

fe4e8290: 00000000 00000000 00000000 00000000 ................

fe4e82a0: 00000000 00000000 00000000 00000000 ................

fe4e82b0: 00000000 00000000 00000000 00000000 ................

fe4e82c0: 00000000 00000000 00000000 00000000 ................

fe4e82d0: 00000000 00000000 00000000 00000000 ................

fe4e82e0: 00000000 00000000 00000000 00000000 ................

fe4e82f0: 00000000 00000000 00000000 00000000 ................

=> md 0xfe4e8300

fe4e8300: 00001002 0000b000 00000000 00000000 ................

fe4e8310: 00000000 00000000 00000000 00000000 ................

fe4e8320: 00000000 00000000 00000000 00000000 ................

fe4e8330: 00000000 00000000 00000000 00000000 ................

fe4e8340: 00000000 00000000 00000000 00000000 ................

fe4e8350: 00000000 00000000 00000000 00000000 ................

fe4e8360: 00000000 00000000 00000000 00000000 ................

fe4e8370: 00000000 00000000 00000000 00000000 ................

fe4e8380: 00000000 00000000 00000000 00000000 ................

fe4e8390: 00000000 00000000 00000000 00000000 ................

fe4e83a0: 00000000 00000000 00000000 00000000 ................

fe4e83b0: 00000000 00000000 00000000 00000000 ................

fe4e83c0: 00000000 00000000 00000000 00000000 ................

fe4e83d0: 00000000 00000000 00000000 00000000 ................

fe4e83e0: 00000000 00000000 00000000 00000000 ................

fe4e83f0: 00000000 00000000 00000000 00000000 ................

0x8E