topic Re: Pcie host reset in T-Series

Pcie host reset

vijayb1 — Sat, 24 Aug 2019 16:42:13 GMT

Hello,

I have programmable FPGA connected on Pcie slot 2, for some reason pcie is in bad state and fails to enumerate device some times. I would like to generate host reset on pcie2 slot, I checked configuration space for this.

25_0000 configuration addr and data register, this is not very clear in the way to generate host reset. Could you please explain how to generate host reset on slot 2?

Thanks

Vijay

Re: Pcie host reset

ufedor — Mon, 26 Aug 2019 09:31:56 GMT

Which processor and which board are in question?

In general initial enumeration will fail if the link to the FPGA is not properly trained.

For additional enumeration it is required:

1) to ensure that the link is properly trained by inspecting the link training status of the specific PCIe controller

2) issue Linux command "/sys/bus/pci/rescan".

Re: Pcie host reset

vijayb1 — Mon, 26 Aug 2019 15:36:54 GMT

This is based T1022E, T1040RDB ref board.

I checked at link training status of Pcie2, it says link down. Is there a way to send host reset using CCSR space.I am not on Linux and i do not have rescan option on my OS.

Thanks

Vijay

Re: Pcie host reset

ufedor — Mon, 26 Aug 2019 17:01:51 GMT

Link down corresponds to a hardware issue.

Please use a digital scope or PCIe analyzer to check the link integrity.

Re: Pcie host reset

vijayb1 — Mon, 26 Aug 2019 17:10:16 GMT

Yes, we started using pcie to collect some more logs on the link state, but assuming if there is an issue with link integrity. Do you think hot reset, would be good idea to bring it to a proper state? and then continue with scan?

Thanks

Vijay

Re: Pcie host reset

ufedor — Mon, 26 Aug 2019 17:24:04 GMT

PCIe controller is trying to establish the link constantly - i.e. there is no need in reset.

Please use a digital scope or PCIe analyzer to check the link integrity.

Re: Pcie host reset

vijayb1 — Mon, 26 Aug 2019 17:26:32 GMT

Thanks will update my findings..

Re: Pcie host reset

vijayb1 — Tue, 27 Aug 2019 23:07:32 GMT

Hello ufedor,

Yes, it is an issue in device not moving to L0 at certain time. On a side note, is it possible to read config space for pcie2 and check link training status bit and poll till it turns to L0 state.

So there are cases. kernel is proceeding with boot sequence before FPGA link training is complete and starts the enumeration.

Is there a way we can poll this bit and halt till training is complete.

I have to admit my understanding on Pcie is not great.

Thanks

Vijay

Re: Pcie host reset

ufedor — Wed, 28 Aug 2019 01:52:12 GMT

Yes. it is possible to poll for the L0 state.

Re: Pcie host reset

vijayb1 — Wed, 28 Aug 2019 02:06:32 GMT

Could you please give me some suggestion to try this out.

From TRM we are on Pcie2 bridge and FPGA is the only EP.

RC->Pcie2->EP

0xcfe250000 is my virtual addr mapping for the physical offset in ccsr space for pcie2.

volatile unsigned int* pci_config = (volatile unsigned int*)0xcfe250000;

volatile unsigned int* pci_data = (volatile unsigned int*)0xcfe250004;

@404 offset we have ltssm register

right this addr in pci config for bus0/devi0/function0.

and check the value from pci_data?

Thanks

Vijay

Re: Pcie host reset

ufedor — Wed, 28 Aug 2019 02:28:40 GMT

Your understanding is correct.

Re: Pcie host reset

vijayb1 — Wed, 28 Aug 2019 20:31:51 GMT

I am still struggling to read the ltssm

#define offset 0x404
volatile unsigned int* pci_config = (volatile unsigned int*)0xcfe250000;
volatile unsigned int* pci_data = (volatile unsigned int*)0xcfe250004;

*pci_config = (0x80000000 | (((offset & 0xf00) << 24)|((0x0<<16))|((0x0<<11))|((0x0<<8))|((offset & 0xff))));
unsigned int data = *pci_data;

data shows value as

data: 100667392

Value is the same for good and bad enumeration. Am i doing it right? and does not add up to link state in TRM.

Re: Pcie host reset

ufedor — Thu, 29 Aug 2019 02:51:11 GMT

> (0x80000000 | (((offset & 0xf00) << 24)|((0x0<<16))|((0x0<<11))|((0x0<<8))|((offset & 0xff))));

What is this value?

Re: Pcie host reset

vijayb1 — Thu, 29 Aug 2019 04:22:56 GMT

Here is the hex value that (I added that to make it more clear)

0x84000004

(10000100000000000000000000000100)

Set ExtReg and Reg Number to 0x404 for BusNumber(0),Device Number(0),Function Number(0)

Re: Pcie host reset

ufedor — Thu, 29 Aug 2019 05:57:53 GMT

Read back the PEX_CONFIG_ADDR register value to ensure that it is written correctly.

> data: 100667392

Seven digits.

Is it decimal?

Re: Pcie host reset

vijayb1 — Thu, 29 Aug 2019 17:55:13 GMT

Yes it is decimal.

I found out that I was on wrong bus. I tried to read some other register like deviceId/VendorId.

0x80010000

I can read vendor Id and Device Id.(0x55551234 - this is correct for our device). Even i cross checked with usb controller which on different control, This works fine.

But when i try to read ltssm it always return 0 (Detect quiet).

My assumption is, if can read device id, link training should be complete and Link should be L0(0x16) state.

Re: Pcie host reset

vijayb1 — Thu, 29 Aug 2019 20:02:28 GMT

If I try to read link status (0x404)for any bus from u-boot.

mw 0xfe240000 0x84010014

md 0xfe240004

mw 0xfe250000 0x84010014

mw 0xfe250004

Always return 0.. So i think extended register i am populating is wrong. :smileysad:

Re: Pcie host reset

ufedor — Fri, 30 Aug 2019 03:02:32 GMT

> If I try to read link status (0x404)for any bus from u-boot.

Please consider that it is senseless to read LTSSM from any bus except the bus 0 (T1022 itself).

Please check that correct PCIe controller is accessed.

Re: Pcie host reset

vijayb1 — Fri, 30 Aug 2019 05:17:22 GMT

Thank you, that makes more sense, and i can see the status now..

Thank you again,

Regards

Vijay