T1042 PCIe Configuration Issue As An Endpoint

[athershehzad]

I am facing an issue related to PCIe configuration on T1042 Power PC, which is configured as an endpoint
Initially my design was working fine with one lane PCIe with 32 bit 2 GB DDR3L, now design has some changes I am looking to configure 4 lane PCIe with 64 bit 4 GB DDR3L.
Output of lspci -vv command on root-complex device gives an unassigned memory in second case. It states BAR addresses to be unassigned.
Comparing from the previous setup output of same command, BAR0 is assigned correctly, while for BAR1 it gives unassigned memory and
for BAR2 it is swapped with the memory assigned to BAR1 in previous case
Below I have attached outputs of both designs
Does anyone know how to solve this issueOutput of New ConfigurationOutput of Old Configuaraion

 @yipingwang  Can you please help. You have solved most my problems earlier too.

[athershehzad]

thanks @ufedor for your kind help. We could not find where this register value was being set, so we hard-coded it.

Now it is working perfectly. Thank you for time and help.

Regards Ather

[ufedor]

Please consider that EP BARs are configured by the EP firmware.

Please provide complete U-Boot and Linux booting logs (as text files attachments) for both configurations.

[athershehzad]

Thanks @ufedor for your prompt reply. Attached are the files that you have requested.

T1042rdb_v1_bootprints.txt represents Old configuration

T1042rdb_v2_bootprints.txt represents new configuration

Regards

Ather

[ufedor]

> I am looking to configure 4 lane PCIe

The same SerDes protocols are used in both configurations and it has PCIe1 x4 trained to x1 Gen1 - i.e. no difference.

Please provide additional information:

1) Output of lspci -vv command on RC device as text files

2) DTS files which are used in both configurations

[athershehzad]

Spoiler

SERDES protocol was set as for 4x Lane PCIe but in both cases we have disconnected 3 Lanes by removing the series capacitors in the lanes. We could not establish 4 lane PCIe connection in the previous version because of design issue but now we can have 4x lane Link, however just for testing purpose, we removed the series capacitors to check for single lane link is working or not.

Here are the files that you have requested.

Dts files were same in both cases

[ufedor]

Is the same DTS used?

Why different parameters for the lspci are provided for the v1:

lspci -vvvv -nn -s 03:00.0

and v2:

lspci -vvvv -nn -s 01:00.0

?

[athershehzad]

Yes same dts is used in both cases.

bus number is changed because in  previous version we were using a PCIe bridge that connects one root complex to two PCIe endpoint out which one endpoint is our T1042. However, in newer version we removed that PCIe bridge and directly connected root complex with endpoint(T1042).

[ufedor]

You wrote:

> I am looking to configure 4 lane PCIe

Why PCIe1 is trained only to x1 Gen1?

[athershehzad]

It is because we have removed capacitors from series path of lane 2, 3 and 4. Hence the only established connection is of single lane PCIe, which must not be a problem since PCIe can downstream itself depending upon the connections.

Also this is done only for debugging purpose.

[ufedor]

Please capture in U-Boot PCIe1 registers (CCSR range 0x24000-0x24F00) dumps for both configurations.

[athershehzad]

Attached txt files are U-Boot PCIe1 registers (CCSR range 0x24000-0x24F00) dumps for both configurations

v2 is for new design 

v1 is for old design

[ufedor]

V1

fe240dc0: 0007e000 00000000 00000000 00000000    ................

fe240dd0: a0f55017 00000000 00000000 00000000    ..P.............

V2

fe240dc0: 000fdff0 00000000 00000000 00000000    ................

fe240dd0: a0f5501f 00000000 00000000 00000000    ..P.............

Why is this difference?

[athershehzad]

This was changed in drivers file for DDR changes (from 2Gb to 4Gb). I thought this change would fix the memory unassigned issue as we assumed that for 4gb ddr we have to set registers accordingly, but it didn't. Forgot to revert the changes. What do you propose I should have done?

[ufedor]

Revert the changes.

[athershehzad]

Output with changes  revert, still same behaviour 

[ufedor · Accepted solution]

Still not reverted:

fe240dd0: a0f5501f 00000000 00000000 00000000 ..P.............

Should be 17