[LX2160ARDB_REV2] PCIe no space for [io size 0x1000]

[tsungmin_wang]

Hi NXP,

We have a PCIe NIC is tested on LSDK 20.04, according to this solution.

https://community.nxp.com/t5/Layerscape/lx2160ardb-rev2-PCIe-card-can-t-be-detected/m-p/1185930

But, LSDK 21.04 is a little different from LSDK 20.04 in dts.

And the kernel showed that no space for [io size 0x1000]

[    2.323441] layerscape-pcie 3700000.pcie: host bridge /soc/pcie@3700000 ranges:
[    2.330757] layerscape-pcie 3700000.pcie:      MEM 0x9840000000..0x987fffffff -> 0x0040000000
[    2.339365] layerscape-pcie 3700000.pcie: PCI host bridge to bus 0000:00
[    2.346060] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.351538] pci_bus 0000:00: root bus resource [mem 0x9840000000-0x987fffffff] (bus address [0x40000000-0x7fffffff])
[    2.362071] pci 0000:00:00.0: [1957:8d90] type 01 class 0x060400
[    2.368132] pci 0000:00:00.0: supports D1 D2
[    2.372394] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot
[    2.379991] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.385224] pci 0000:00:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    2.393774] layerscape-pcie 3800000.pcie: host bridge /soc/pcie@3800000 ranges:
[    2.401099] layerscape-pcie 3800000.pcie:      MEM 0xa040000000..0xa07fffffff -> 0x0040000000
[    2.409707] layerscape-pcie 3800000.pcie: PCI host bridge to bus 0001:00
[    2.416402] pci_bus 0001:00: root bus resource [bus 00-ff]
[    2.421881] pci_bus 0001:00: root bus resource [mem 0xa040000000-0xa07fffffff] (bus address [0x40000000-0x7fffffff])
[    2.432408] pci 0001:00:00.0: [1957:8d90] type 01 class 0x060400
[    2.438483] pci 0001:00:00.0: supports D1 D2
[    2.442745] pci 0001:00:00.0: PME# supported from D0 D1 D2 D3hot
[    2.449918] pci 0001:01:00.0: [10b5:8724] type 01 class 0x060400
[    2.456102] pci 0001:01:00.0: reg 0x10: [mem 0xa040000000-0xa04003ffff]
[    2.464287] pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
[    2.472701] pci 0001:02:00.0: [10b5:8724] type 01 class 0x060400
[    2.480505] pci 0001:02:00.0: PME# supported from D0 D3hot D3cold
[    2.487738] pci 0001:02:01.0: [10b5:8724] type 01 class 0x060400
[    2.495541] pci 0001:02:01.0: PME# supported from D0 D3hot D3cold
[    2.502990] pci 0001:02:08.0: [10b5:8724] type 01 class 0x060400
[    2.510795] pci 0001:02:08.0: PME# supported from D0 D3hot D3cold
[    2.518900] pci_bus 0001:03: busn_res: can not insert [bus 03] under [bus 02] (conflicts with (null) [bus 02])
[    2.529622] pci 0001:02:00.0: devices behind bridge are unusable because [bus 03] cannot be assigned for them
[    2.539887] pci_bus 0001:04: busn_res: can not insert [bus 04] under [bus 02] (conflicts with (null) [bus 02])
[    2.550604] pci 0001:02:01.0: devices behind bridge are unusable because [bus 04] cannot be assigned for them
[    2.560870] pci_bus 0001:05: busn_res: can not insert [bus 05] under [bus 02] (conflicts with (null) [bus 02])
[    2.571588] pci 0001:02:08.0: devices behind bridge are unusable because [bus 05] cannot be assigned for them
[    2.581682] pci 0001:01:00.0: bridge has subordinate 02 but max busn 05
[    2.588373] pci 0001:00:00.0: BAR 14: assigned [mem 0xa040000000-0xa0404fffff]
[    2.595589] pci 0001:00:00.0: BAR 15: assigned [mem 0xa040500000-0xa0408fffff 64bit pref]
[    2.603757] pci 0001:00:00.0: BAR 13: no space for [io  size 0x2000]
[    2.610103] pci 0001:00:00.0: BAR 13: failed to assign [io  size 0x2000]
[    2.616797] pci 0001:01:00.0: BAR 14: assigned [mem 0xa040000000-0xa0403fffff]
[    2.624011] pci 0001:01:00.0: BAR 15: assigned [mem 0xa040500000-0xa0408fffff 64bit pref]
[    2.632179] pci 0001:01:00.0: BAR 0: assigned [mem 0xa040400000-0xa04043ffff]
[    2.639334] pci 0001:01:00.0: BAR 13: no space for [io  size 0x2000]
[    2.645679] pci 0001:01:00.0: BAR 13: failed to assign [io  size 0x2000]
[    2.652374] pci 0001:02:00.0: BAR 14: assigned [mem 0xa040000000-0xa0401fffff]
[    2.659588] pci 0001:02:00.0: BAR 15: assigned [mem 0xa040500000-0xa0406fffff 64bit pref]
[    2.667756] pci 0001:02:01.0: BAR 14: assigned [mem 0xa040200000-0xa0403fffff]
[    2.674970] pci 0001:02:01.0: BAR 15: assigned [mem 0xa040700000-0xa0408fffff 64bit pref]
[    2.683138] pci 0001:02:00.0: BAR 13: no space for [io  size 0x1000]
[    2.689483] pci 0001:02:00.0: BAR 13: failed to assign [io  size 0x1000]
[    2.696175] pci 0001:02:01.0: BAR 13: no space for [io  size 0x1000]
[    2.702519] pci 0001:02:01.0: BAR 13: failed to assign [io  size 0x1000]
[    2.709211] pci 0001:02:00.0: PCI bridge to [bus 03]
[    2.714209] pci 0001:02:00.0:   bridge window [mem 0xa040000000-0xa0401fffff]
[    2.721365] pci 0001:02:00.0:   bridge window [mem 0xa040500000-0xa0406fffff 64bit pref]
[    2.729502] pci 0001:02:01.0: PCI bridge to [bus 04]
[    2.734501] pci 0001:02:01.0:   bridge window [mem 0xa040200000-0xa0403fffff]
[    2.741655] pci 0001:02:01.0:   bridge window [mem 0xa040700000-0xa0408fffff 64bit pref]
[    2.749792] pci 0001:02:08.0: PCI bridge to [bus 05]
[    2.754874] pci 0001:01:00.0: PCI bridge to [bus 02]
[    2.759874] pci 0001:01:00.0:   bridge window [mem 0xa040000000-0xa0403fffff]
[    2.767028] pci 0001:01:00.0:   bridge window [mem 0xa040500000-0xa0408fffff 64bit pref]
[    2.775165] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    2.780385] pci 0001:00:00.0:   bridge window [mem 0xa040000000-0xa0404fffff]
[    2.787512] pci 0001:00:00.0:   bridge window [mem 0xa040500000-0xa0408fffff 64bit pref]
[    2.795606] pci 0001:00:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    2.804177] pci 0001:01:00.0: Max Payload Size set to  256/2048 (was  128), Max Read Rq  128
[    2.812760] pci 0001:02:00.0: Max Payload Size set to  256/2048 (was  128), Max Read Rq  128
[    2.821344] pci 0001:02:01.0: Max Payload Size set to  256/2048 (was  128), Max Read Rq  128
[    2.829927] pci 0001:02:08.0: Max Payload Size set to  256/2048 (was  128), Max Read Rq  128

We think this is a same problem, but need the help to modify dts.

I have tried to increase configuration space like this to all pcie

<0x80 0x00000000 0x0 0x00002000> => <0x80 0x00000000 0x0 0x00020000>

But it doesn't work, still no space to [io size 0x1000]

[yipingwang]

Does your EP device need to use IO space?

If no, this will not affect the function.

[tsungmin_wang]

Hi,

This is Broadcom/PLX PEX8724 6-port PCIe Switch with 4 Intel 82599ES 10G SFI/SFP+ NIC connected underneath the PLX downstream port.

I'm not sure about the IO space.

But the previous solotion is increasing the address size in dts to PCIe (uboot and linux).

I can get Broadcom/PLX PEX8724 6-port PCIe Switch information, but 4 Intel 82599ES 10G SFI/SFP+ NIC is not showed when I use lspci.

It should look like this

0001:00:00.0 PCI bridge [0604]: Device [xxxx:xxxx] 
0001:01:00.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:02:00.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:02:01.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:02:08.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:03:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
0001:03:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
0001:04:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
0001:04:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

But I only get

0001:00:00.0 PCI bridge [0604]: Device [xxxx:xxxx] 
0001:01:00.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:02:00.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:02:01.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)
0001:02:08.0 PCI bridge [0604]: PLX Technology, Inc. Device [10b5:8724] (rev ca)

[yipingwang]

I discussed with the LSDK development team, 4 Intel 82599ES 10G SFI/SFP+ NIC is not showed when using lspci, this is should not be caused by lack of IO space.

I need to discuss with the AE team, will provide feedback later.

[yipingwang · Accepted solution]

Please check whether the following u-boot patch could be used to resolve your problem.

https://source.denx.de/u-boot/u-boot/-/commit/19e1b8d9b283514eb6ab8d7253f4867068007938

[tsungmin_wang]

Hi, 

Thanks, the issue is solved.