T1023 PCIe issues

[bitcom]

I am using T1023 with RCW set to serdes (0x99) , uboot boot up shows

 Corenet Platform Cache: 256 KiB enabled
Using SERDES1 Protocol: 153 (0x99)
SEC0: RNG instantiated
NAND: 0 MiB
MMC: FSL_SDHC: 0
SF: Detected S25FL512S_256K with page size 256 Bytes, erase size 256 KiB, total 64 MiB
EEPROM: Invalid ID (ff ff ff ff)
PCIe1: Root Complex, no link, regs @ 0xfe240000
PCIe1: Bus 00 - 00
PCIe2: disabled
PCIe3: Root Complex, no link, regs @ 0xfe260000
PCIe3: Bus 01 - 01

my questions are 

1. why no link shows here? my ITSSM value is 0. (I got a PCIe device connected...)

2. I observe the same log "no link" on the T1023 reference board, is ITSSM supposed to be 0 even with no PCIe device?

3. Could you show me how the ITSSM process is triggered, is there an enable register of the PCIe controller?

Thanks,

ZZ

[ufedor]

1) Check the link integrity including AC coupling capacitors - they should be 100 nF.

Check that SerDes reference clocks are stable before PORESET_B is deasserted.

2) Refer to the QorIQ T1024 Reference Manual, 27.11.16 LTSSM State Status Register (LTSSM_State_Status_Register)

LTSSM=0x00 - no link partner detected

LTSSM=0x16 - L0, link is properly trained

3) The training process is continuous (no need to trigger) after POR for the PCIe lane

You wrote:

> I observe the same log "no link" on the T1023 reference board

It could be that you've used incorrect method because the LTSSM State Status Register is not memory-mapped - it resides in the PCI Express Extended Configuration Space.

[bitcom]

Thanks for your reply. 

To be more specific, I observe the followings on the reference board

=> pci header 0
vendor ID = 0x1957
device ID = 0x0848
command register = 0x0000
status register = 0x0010
revision ID = 0x10
class code = 0x0b (Processor)
sub class code = 0x20
programming interface = 0x00
cache line = 0x00
latency time = 0x00
header type = 0x01
BIST = 0x00
base address 0 = 0xff000000
base address 1 = 0x00000000
primary bus number = 0x00
secondary bus number = 0x00
subordinate bus number = 0x00
secondary latency timer = 0x00
IO base = 0x01
IO limit = 0x01
secondary status = 0x0000
memory base = 0x0000
memory limit = 0x0000
prefetch memory base = 0x0001
prefetch memory limit = 0x0001
prefetch memory base upper = 0x00000000
prefetch memory limit upper = 0x00000000
IO base upper 16 bits = 0x0000
IO limit upper 16 bits = 0x0000
expansion ROM base address = 0x00000000
interrupt line = 0x00
interrupt pin = 0x00
bridge control = 0x0000
=> pci display 0.0.0 404
00000404: 00000000 000004e2 00000000 00000100
00000414: 00000000 00000000 96964040 00000000
00000424: 00000000 00000000 00000000 00000000
00000434: 00000000 00c2415c 00000000 00000010
=>

There is no PCIe device connected on Reference board... But I assume the LTSSM should not be 0???

BTW, the dump info the same on our board with the Reference board.

Thanks

[ufedor]

> But I assume the LTSSM should not be 0???

The PCIe link could be trained only when properly operating EP device is connected to the RC.

> I read  the value of register 0xfe0ea8c0 is 0x11010000 which seems to be incorrect???

In this register there are many fields which are overridden by the protocol logic when running in the PCIe mode.

[bitcom]

So you mean our EP device is not functioning correctly, thus the link trainning is not triggered? 

I can detect the EP device keeps sending out signal on our RX line. Can you give me some suggestion on how to further debug this in order to determine weather EP is not functioning or the RC setting is not correct?

Thanks a lot.

[ufedor]

If you are confident that AC coupling capacitors are 100nF it is additionally needed to ensure that the EP has 50 Ohm termination on each differential RX signal.

[bitcom]

Hi ufedor,

my RCW setting is 

Reset Configuration Word (RCW):
00000000: 0810000c 00000000 00000000 00000000
00000010: 4c800003 80000012 58104000 21000000
00000020: 00000000 00000000 00000000 00022800
00000030: 00000530 08020200 00000000 00000006
Board: T1023RDB, RevB, boot from SPI
SERDES Reference Clocks:
SD1_CLK1=100.00MHZ, SD1_CLK2=125.00MHz
I2C: ready
SPI: ready
DRAM: starting at step 1 (STEP_GET_SPD)
Filling dimm parameters from board specific file
Computing lowest common DIMM parameters for memctl=0
Detected UDIMM Fixed DDR4 on board
Reloading memory controller configuration options for memctl=0
dbw_cap_adj[0]=0
ctrl 0 dimm 0 base 0x0
ctrl 0 total 0x80000000
Total mem by __step_assign_addresses is 0x80000000
Total mem 2147483648 assigned
FSL Memory ctrl register computation
2 GiB (DDR4, 32-bit, CL=11, ECC off)
Flash: 0 Bytes
L2: 256 KiB enabled
Corenet Platform Cache: 256 KiB enabled
Using SERDES1 Protocol: 153 (0x99)

I am using T1023rdb's code , T1023rdb is using SERDES1 portocl 0x77,  besides change RCW's SERDES1 to Protocol: 153 (0x99), what other software settings are required in order for PCIe1 (LANE D) to work well??

Thanks 

[bitcom]

whats the steps to check the settings of PCIe1 (LANE D) on board is correct?

Thanks

[ufedor · Accepted solution]

1) What is the SerDes_PCCR0 value?

2) What is the SerDes PLL2 reference clocks frequency?

If it is really 125MHz, then RCW[161] setting is incorrect - it should be 1 (higher frequency reference clock).

[bitcom]

It turns out our PLL frequency is not correct. after selecting the 100MHz  PLL everything works fine. 

[bitcom]

I checked followings are fine...

Check the link integrity including AC coupling capacitors - they should be 100 nF.

Check that SerDes reference clocks are stable before PORESET_B is deasserted.

I read  the value of register 0xfe0ea8c0 is 0x11010000 which seems to be incorrect???