PCIe Host Bridges on MPC8572E

[rudolfstreif]

I am in process of creating Linux platform support for a board that uses the MPC8572E processor with the 36-bit bus. The Linux kernel version is 3.10. I am building with the Yocto project. The kernel boots fine with the device tree. However, I am having difficulties getting the PCIe host bridges to work. They seem to get probed and initialized by the code in arch/powerpc/sysdev/fsl_pci.c (see log below). However, the host bridges never show up /proc/bus/pci/devices and /sys/bus/pci.

What also strike me odd is that for all three bridges PCICSRBAR is 0x0 and the DMA window size is 0x0.

Any help to further debug would be appreciated.

Thanks,

Rudi

Adding PCI host bridge /pcie@fffe08000

PCI memory map start 0x0000000fffe08000, size 0x0000000000001000

Found FSL PCI host bridge at 0x0000000fffe08000. Firmware bus number: 0->255

->Hose at 0xef163a00, cfg_addr=0xf1016000,cfg_data=0xf1016004

PCI host bridge /pcie@fffe08000 (primary) ranges:

MEM 0x0000000c00000000..0x0000000c1fffffff -> 0x00000000e0000000

  IO 0x0000000fffc00000..0x0000000fffc0ffff -> 0x0000000000000000

PCI MEM resource start 0x0000000c00000000, size 0x0000000020000000.

PCI IO resource start 0x0000000000000000, size 0x0000000000010000, phy base 0x0000000fffc00000.

/pcie@fffe08000: PCICSRBAR @ 0x0

/pcie@fffe08000: DMA window size is 0x0

Adding PCI host bridge /pcie@fffe09000

PCI memory map start 0x0000000fffe09000, size 0x0000000000001000

Found FSL PCI host bridge at 0x0000000fffe09000. Firmware bus number: 0->255

->Hose at 0xef182000, cfg_addr=0xf101a000,cfg_data=0xf101a004

PCI host bridge /pcie@fffe09000  ranges:

MEM 0x0000000c20000000..0x0000000c3fffffff -> 0x00000000e0000000

  IO 0x0000000fffc10000..0x0000000fffc1ffff -> 0x0000000000000000

PCI MEM resource start 0x0000000c20000000, size 0x0000000020000000.

PCI IO resource start 0x0000000000000000, size 0x0000000000010000, phy base 0x0000000fffc10000.

/pcie@fffe09000: PCICSRBAR @ 0x0

/pcie@fffe09000: DMA window size is 0x0

Adding PCI host bridge /pcie@fffe0A000

PCI memory map start 0x0000000fffe0a000, size 0x0000000000001000

Found FSL PCI host bridge at 0x0000000fffe0a000. Firmware bus number: 0->255

->Hose at 0xef182600, cfg_addr=0xf101e000,cfg_data=0xf101e004

PCI host bridge /pcie@fffe0A000  ranges:

MEM 0x0000000c40000000..0x0000000c5fffffff -> 0x00000000e0000000

  IO 0x0000000fffc20000..0x0000000fffc2ffff -> 0x0000000000000000

PCI MEM resource start 0x0000000c40000000, size 0x0000000020000000.

PCI IO resource start 0x0000000000000000, size 0x0000000000010000, phy base 0x0000000fffc20000.

/pcie@fffe0A000: PCICSRBAR @ 0x0

/pcie@fffe0A000: DMA window size is 0x0

gpiochip_find_base: found new base at 224

gpiochip_add: registered GPIOs 224 to 255 on device: /soc8572@fffe00000/gpio-controller@f000

software IO TLB [mem 0x050dd000-0x090dd000] (64MB) mapped at [c50dd000-c90dcfff]

PCI: Probing PCI hardware

fsl-pci fffe08000.pcie: PCI host bridge to bus 0000:00

pci_bus 0000:00: root bus resource [io  0x0000-0xffff]

pci_bus 0000:00: root bus resource [mem 0xc00000000-0xc1fffffff] (bus address [0xe0000000-0xffffffff])

pci_bus 0000:00: root bus resource [bus 00-ff]

pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff

pci_bus 0000:00: scanning bus

pci_bus 0000:00: fixups for bus

pci_bus 0000:00: bus scan returning with max=00

pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00

fsl-pci fffe09000.pcie: PCI host bridge to bus 0001:01

pci_bus 0001:01: root bus resource [io  0x20000-0x2ffff] (bus address [0x0000-0xffff])

pci_bus 0001:01: root bus resource [mem 0xc20000000-0xc3fffffff] (bus address [0xe0000000-0xffffffff])

pci_bus 0001:01: root bus resource [bus 01-ff]

pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to ff

pci_bus 0001:01: scanning bus

pci_bus 0001:01: fixups for bus

pci_bus 0001:01: bus scan returning with max=01

pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to 01

fsl-pci fffe0a000.pcie: PCI host bridge to bus 0002:02

pci_bus 0002:02: root bus resource [io  0x40000-0x4ffff] (bus address [0x0000-0xffff])

pci_bus 0002:02: root bus resource [mem 0xc40000000-0xc5fffffff] (bus address [0xe0000000-0xffffffff])

pci_bus 0002:02: root bus resource [bus 02-ff]

pci_bus 0002:02: busn_res: [bus 02-ff] end is updated to ff

pci_bus 0002:02: scanning bus

pci_bus 0002:02: fixups for bus

pci_bus 0002:02: bus scan returning with max=02

pci_bus 0002:02: busn_res: [bus 02-ff] end is updated to 02

pci_bus 0000:00: resource 4 [io  0x0000-0xffff]

pci_bus 0000:00: resource 5 [mem 0xc00000000-0xc1fffffff]

pci_bus 0001:01: resource 4 [io  0x20000-0x2ffff]

pci_bus 0001:01: resource 5 [mem 0xc20000000-0xc3fffffff]

pci_bus 0002:02: resource 4 [io  0x40000-0x4ffff]

pci_bus 0002:02: resource 5 [mem 0xc40000000-0xc5fffffff]

[scottwood]

Could you enable pr_debug() output in arch/powerpc/sysdev/fsl_pci.c (e.g. after the last #include, place the line "#define pr_debug pr_info")?

Also, in setup_pci_atmu() immediately after the early_read_config_word() call, add:

     pr_info("paddr_lo %llx paddr_hi %llx pcicsrbar_sz %x\n", paddr_lo, paddr_hi, pcicsrbar_sz);

[rudolfstreif]

Thank you for your support, Scott. I very much appreciate it. Did as you asked. Please see output below. There are some additional pr_debug messages that I had added to understand the flow of things.

Thanks,

Rudi

fsl_pci_probe: entry

Adding PCI host bridge /pcie@fffe08000

PCI memory map start 0x0000000fffe08000, size 0x0000000000001000

Found FSL PCI host bridge at 0x0000000fffe08000. Firmware bus number: 0->255

->Hose at 0xef163a00, cfg_addr=0xf1016000,cfg_data=0xf1016004

PCI host bridge /pcie@fffe08000 (primary) ranges:

MEM 0x0000000c00000000..0x0000000c1fffffff -> 0x00000000e0000000

  IO 0x0000000fffc00000..0x0000000fffc0ffff -> 0x0000000000000000

setup_pci_atmu: entry

PCI MEM resource start 0x0000000c00000000, size 0x0000000020000000.

PCI IO resource start 0x0000000000000000, size 0x0000000000010000, phy base 0x0000000fffc00000.

paddr_lo e0000000 paddr_hi ffffffff pcicsrbar_sz 0

/pcie@fffe08000: PCICSRBAR @ 0x0

end of RAM: 0x0000000200000000

/pcie@fffe08000: DMA window size is 0x0

fsl_pci_probe: entry

Adding PCI host bridge /pcie@fffe09000

PCI memory map start 0x0000000fffe09000, size 0x0000000000001000

Found FSL PCI host bridge at 0x0000000fffe09000. Firmware bus number: 0->255

->Hose at 0xef182000, cfg_addr=0xf101a000,cfg_data=0xf101a004

PCI host bridge /pcie@fffe09000  ranges:

MEM 0x0000000c20000000..0x0000000c3fffffff -> 0x00000000e0000000

  IO 0x0000000fffc10000..0x0000000fffc1ffff -> 0x0000000000000000

setup_pci_atmu: entry

PCI MEM resource start 0x0000000c20000000, size 0x0000000020000000.

PCI IO resource start 0x0000000000000000, size 0x0000000000010000, phy base 0x0000000fffc10000.

paddr_lo e0000000 paddr_hi ffffffff pcicsrbar_sz 0

/pcie@fffe09000: PCICSRBAR @ 0x0

end of RAM: 0x0000000200000000

/pcie@fffe09000: DMA window size is 0x0

fsl_pci_probe: entry

Adding PCI host bridge /pcie@fffe0A000

PCI memory map start 0x0000000fffe0a000, size 0x0000000000001000

Found FSL PCI host bridge at 0x0000000fffe0a000. Firmware bus number: 0->255

->Hose at 0xef182600, cfg_addr=0xf101e000,cfg_data=0xf101e004

PCI host bridge /pcie@fffe0A000  ranges:

MEM 0x0000000c40000000..0x0000000c5fffffff -> 0x00000000e0000000

  IO 0x0000000fffc20000..0x0000000fffc2ffff -> 0x0000000000000000

setup_pci_atmu: entry

PCI MEM resource start 0x0000000c40000000, size 0x0000000020000000.

PCI IO resource start 0x0000000000000000, size 0x0000000000010000, phy base 0x0000000fffc20000.

paddr_lo e0000000 paddr_hi ffffffff pcicsrbar_sz 0

/pcie@fffe0A000: PCICSRBAR @ 0x0

end of RAM: 0x0000000200000000

/pcie@fffe0A000: DMA window size is 0x0

gpiochip_find_base: found new base at 224

gpiochip_add: registered GPIOs 224 to 255 on device: /soc8572@fffe00000/gpio-controller@f000

software IO TLB [mem 0x050dd000-0x090dd000] (64MB) mapped at [c50dd000-c90dcfff]

PCI: Probing PCI hardware

fsl-pci fffe08000.pcie: PCI host bridge to bus 0000:00

pci_bus 0000:00: root bus resource [io  0x0000-0xffff]

pci_bus 0000:00: root bus resource [mem 0xc00000000-0xc1fffffff] (bus address [0xe0000000-0xffffffff])

pci_bus 0000:00: root bus resource [bus 00-ff]

pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to ff

pci_bus 0000:00: scanning bus

pci_bus 0000:00: fixups for bus

fsl_pcibios_fixup_bus: entry

pci_bus 0000:00: bus scan returning with max=00

pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00

fsl-pci fffe09000.pcie: PCI host bridge to bus 0001:01

pci_bus 0001:01: root bus resource [io  0x20000-0x2ffff] (bus address [0x0000-0xffff])

pci_bus 0001:01: root bus resource [mem 0xc20000000-0xc3fffffff] (bus address [0xe0000000-0xffffffff])

pci_bus 0001:01: root bus resource [bus 01-ff]

pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to ff

pci_bus 0001:01: scanning bus

pci_bus 0001:01: fixups for bus

fsl_pcibios_fixup_bus: entry

pci_bus 0001:01: bus scan returning with max=01

pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to 01

fsl-pci fffe0a000.pcie: PCI host bridge to bus 0002:02

pci_bus 0002:02: root bus resource [io  0x40000-0x4ffff] (bus address [0x0000-0xffff])

pci_bus 0002:02: root bus resource [mem 0xc40000000-0xc5fffffff] (bus address [0xe0000000-0xffffffff])

pci_bus 0002:02: root bus resource [bus 02-ff]

pci_bus 0002:02: busn_res: [bus 02-ff] end is updated to ff

pci_bus 0002:02: scanning bus

pci_bus 0002:02: fixups for bus

fsl_pcibios_fixup_bus: entry

pci_bus 0002:02: bus scan returning with max=02

pci_bus 0002:02: busn_res: [bus 02-ff] end is updated to 02

pci_bus 0000:00: resource 4 [io  0x0000-0xffff]

pci_bus 0000:00: resource 5 [mem 0xc00000000-0xc1fffffff]

pci_bus 0001:01: resource 4 [io  0x20000-0x2ffff]

pci_bus 0001:01: resource 5 [mem 0xc20000000-0xc3fffffff]

pci_bus 0002:02: resource 4 [io  0x40000-0x4ffff]

pci_bus 0002:02: resource 5 [mem 0xc40000000-0xc5fffffff]

[rudolfstreif]

I have been digging some more. In the function setup_pci_atmu() in sysdev/fsl_pci.c there is the following code to detect if the PCI bridge is a PCIe bridge:

	/* PCIe can overmap inbound & outbound since RX & TX are separated */
	if (early_find_capability(hose, 0, 0, PCI_CAP_ID_EXP)) { ... }

The CPU our board is using is a MPC8572E. For this CPU I would expect the call to early_find_capability(hose, 0, 0, PCI_CAP_EXP) to return a non-zero value. However, it does not. Any reason why it would not?

Thanks,

Rudi