LX2160A SGMII PCS not responding on MDIO bus

[equinox]

Hi all,

I'm trying to configure a LX2160A (on the Solid-Run CEx7) for SGMII operation. However, trying to access the SGMII PCS (addr 3) on the MDIO interface returns only zeros. Same for the backplane AN (addr 7) and vendor-specific (addr 30) management ports. Whatever I try, only 0 is returned.

In USXGMII/XFI configuration (on SERDES 1), it is working, so the MDIO interface itself seems to be operating correctly.

Also the SERDES is operating at 1G correctly, I'm seeing a link being established (though it cannot exchange any data it seems, but that might be expected when unable to access PCS management.)

My primary target is configuration 13 on SERDES2 using MACs 13 & 14. I've made sure to edit the DPC to specify SGMII mode. I've updated MC firmware to 10.30.0 in case something was fixed, but no change. I've also tried switching SERDES1 to configuration 6 or 7, same problem.

Does anyone know if there is some further configuration/setting necessary to get access to SGMII PCS on MDIO? (Note: the instructions in reference manual 26.10.1.1 have no effect - they use the MDIO interface, which doesn't seem to work! I've tried anyway, but nope...)

Cheers,

equi

[equinox · Accepted solution]

(repost, my earlier post disappeared for some reason)

Found the problem - apparently SGMIIx_KX must be set for the SERDES lane in PCC8 (i.e. KX mode instead of SGMII, cf. reference manual 26.4.1.34) to make device 3/7/30 MDIO registers accessible:

=> mii device mdio@8c37000 
=> echo SERDES2@1eb0000, lane G/6=DPMAC13 H/7=DPMAC14, config 13
SERDES2@1eb0000, lane G/6=DPMAC13 H/7=DPMAC14, config 13
=> md.l 1eb10a0 1
01eb10a0: 00000011                               ....
=> mii read 0 0-f
addr=00 reg=00 data=1140
addr=00 reg=01 data=0029
addr=00 reg=02 data=0083
addr=00 reg=03 data=E400
addr=00 reg=04 data=4001
addr=00 reg=05 data=40A0
addr=00 reg=06 data=0006
addr=00 reg=07 data=0000
addr=00 reg=08 data=0000
addr=00 reg=09 data=0000
addr=00 reg=0a data=0000
addr=00 reg=0b data=0000
addr=00 reg=0c data=0000
addr=00 reg=0d data=0000
addr=00 reg=0e data=0000
addr=00 reg=0f data=0000
=> mii read 0 0-f devad 3
addr=00 devad=03 reg=00 data=0000
addr=00 devad=03 reg=01 data=0000
addr=00 devad=03 reg=02 data=0000
addr=00 devad=03 reg=03 data=0000
addr=00 devad=03 reg=04 data=0000
addr=00 devad=03 reg=05 data=0000
addr=00 devad=03 reg=06 data=0000
addr=00 devad=03 reg=07 data=0000
addr=00 devad=03 reg=08 data=0000
addr=00 devad=03 reg=09 data=0000
addr=00 devad=03 reg=0a data=0000
addr=00 devad=03 reg=0b data=0000
addr=00 devad=03 reg=0c data=0000
addr=00 devad=03 reg=0d data=0000
addr=00 devad=03 reg=0e data=0000
addr=00 devad=03 reg=0f data=0000
=> echo reconfigure to KX mode (PCC8)
reconfigure to KX mode (PCC8)
=> mw.l 1eb10a0 99
=> md.l 1eb10a0 1
01eb10a0: 00000099                               ....
=> mii read 0 0-f
addr=00 reg=00 data=1140
addr=00 reg=01 data=002D
addr=00 reg=02 data=0083
addr=00 reg=03 data=E400
addr=00 reg=04 data=4001
addr=00 reg=05 data=40A0
addr=00 reg=06 data=0004
addr=00 reg=07 data=0000
addr=00 reg=08 data=0000
addr=00 reg=09 data=0000
addr=00 reg=0a data=0000
addr=00 reg=0b data=0000
addr=00 reg=0c data=0000
addr=00 reg=0d data=0000
addr=00 reg=0e data=0000
addr=00 reg=0f data=0000
=> mii read 0 0-f devad 3            
addr=00 devad=03 reg=00 data=1140
addr=00 devad=03 reg=01 data=002D
addr=00 devad=03 reg=02 data=0083
addr=00 devad=03 reg=03 data=E400
addr=00 devad=03 reg=04 data=0000
addr=00 devad=03 reg=05 data=0088
addr=00 devad=03 reg=06 data=0000
addr=00 devad=03 reg=07 data=0000
addr=00 devad=03 reg=08 data=0000
addr=00 devad=03 reg=09 data=0000
addr=00 devad=03 reg=0a data=0000
addr=00 devad=03 reg=0b data=0000
addr=00 devad=03 reg=0c data=0000
addr=00 devad=03 reg=0d data=0000
addr=00 devad=03 reg=0e data=0083
addr=00 devad=03 reg=0f data=E400

This is extremely confusing since one would really expect the SGMII PCS to be accessible in SGMII mode(?!?)

On the plus side, however, everything (including data exchange) seems to work after configuring the lane to KX mode instead of SGMII mode.

@ufedorcan you please investigate what is going on here? Is the documentation inverted and 1 = SGMII / 0 = KX mode? Or is there some errata in the silicon? Since SGMII protocol can be turned on/off in PCS IF_MODE register, is there any reason to have SGMIIx_KX = 0 in PCC8?

-equi

[equinox]

Found the problem - the SGMII PCS/Backplane AN/Vendor MDIO registers are accessible only if SGMIIx_KX bit is set in SERDES PCC8 register:

=> mii device mdio@8c37000
=> md.l 1eb10a0
01eb10a0: 00000011                               ....
=> mii read 0 0-f
addr=00 reg=00 data=1140
addr=00 reg=01 data=002D
addr=00 reg=02 data=0083
addr=00 reg=03 data=E400
addr=00 reg=04 data=4001
addr=00 reg=05 data=40A0
addr=00 reg=06 data=0004
addr=00 reg=07 data=0000
addr=00 reg=08 data=0000
addr=00 reg=09 data=0000
addr=00 reg=0a data=0000
addr=00 reg=0b data=0000
addr=00 reg=0c data=0000
addr=00 reg=0d data=0000
addr=00 reg=0e data=0000
addr=00 reg=0f data=0000
=> mii read 0 0-f devad 3
addr=00 devad=03 reg=00 data=0000
addr=00 devad=03 reg=01 data=0000
addr=00 devad=03 reg=02 data=0000
addr=00 devad=03 reg=03 data=0000
addr=00 devad=03 reg=04 data=0000
addr=00 devad=03 reg=05 data=0000
addr=00 devad=03 reg=06 data=0000
addr=00 devad=03 reg=07 data=0000
addr=00 devad=03 reg=08 data=0000
addr=00 devad=03 reg=09 data=0000
addr=00 devad=03 reg=0a data=0000
addr=00 devad=03 reg=0b data=0000
addr=00 devad=03 reg=0c data=0000
addr=00 devad=03 reg=0d data=0000
addr=00 devad=03 reg=0e data=0000
addr=00 devad=03 reg=0f data=0000

###### change PCC8 to KX mode

=> mw.l 1eb10a0 99       
=> md.l 1eb10a0          
01eb10a0: 00000099                               ....
=> mii read 0 0-f        
addr=00 reg=00 data=1140
addr=00 reg=01 data=002D
addr=00 reg=02 data=0083
addr=00 reg=03 data=E400
addr=00 reg=04 data=4001
addr=00 reg=05 data=40A0
addr=00 reg=06 data=0004
addr=00 reg=07 data=0000
addr=00 reg=08 data=0000
addr=00 reg=09 data=0000
addr=00 reg=0a data=0000
addr=00 reg=0b data=0000
addr=00 reg=0c data=0000
addr=00 reg=0d data=0000
addr=00 reg=0e data=0000
addr=00 reg=0f data=0000
=> mii read 0 0-f devad 3
addr=00 devad=03 reg=00 data=1140
addr=00 devad=03 reg=01 data=002D
addr=00 devad=03 reg=02 data=0083
addr=00 devad=03 reg=03 data=E400
addr=00 devad=03 reg=04 data=0000
addr=00 devad=03 reg=05 data=0088
addr=00 devad=03 reg=06 data=0000
addr=00 devad=03 reg=07 data=0000
addr=00 devad=03 reg=08 data=0000
addr=00 devad=03 reg=09 data=0000
addr=00 devad=03 reg=0a data=0000
addr=00 devad=03 reg=0b data=0000
addr=00 devad=03 reg=0c data=0000
addr=00 devad=03 reg=0d data=0000
addr=00 devad=03 reg=0e data=0083
addr=00 devad=03 reg=0f data=E400

(note the "devad" option on the mii command is a custom addition)

Now the real question is, why is the SGMII PCS accessible only when in KX mode, but not in SGMII mode?!? Something seems wrong somewhere here. Is the documentation wrong/inverted on the KX bit?

@ufedorcan you please investigate WTF is going on here?

[ufedor]

Please consider that the board in question is proprietary so it is needed to request support directly from SolidRun at first.

In case the SolidRun techsupport analysis will indicate that the issue is solely caused by the LX2160A, then detailed technical description of the issue (including processor connection schematics and all necessary logs) should be provided.

[equinox]

Hi @ufedor 

this issue is about internal circuitry of the LX2160A, the techsupport analyis is done already. To clarify, I'm asking regarding accessing the *on-chip* SGMII PCS through the *on-chip* MDIO interface. No board signals are involved at all.

As far as logs are concerned, I'm simply trying to access MDIO registers - reproduced in u-boot here to eliminate other possible sources of problems:

=> echo addr_write; mw.l 0x8c37030 40009548; md.l 0x8c37030 1; mw.l 0x8c37034 3; mw.l 0x8c3703c 4; md.l 0x8c37030 1
addr_write
00009548
40009548
=> echo reg_read; mw.l 0x8c37030 40009548; md.l 0x8c37030 1; mw.l 0x8c37034 3; mw.l 0x8c37034 0x8003; md.l 0x8c37030 1; md.l 0x8c37038 1
reg_read
00009548
40009548
00000000

The expected result (last line) is 000001A0 according to reference manual 26.9.2.1.1.1.

In the meantime I have noticed that there is a response on Clause 22 access, but it does not match any documented interface in the reference manual(?!?)

Can you please clarify what the necessary sequence is to access the PCS / Backplane AN / Vendor specific registers for the SGMII interface? The instructions in 26.10.1.1 seem to be insufficient (it doesn't seem possible to access the SGMII IF Mode register to begin with.) Is there some further initialization needed? Does the SGMII interface not support Clause 45 access/what access method should be used? (The reference manual is extremely unspecific on this!)

Alternatively, can you please confirm 1G SGMII works on the LX2160A RDB and point me to example initialization/setup code?

equi