ls1021a: MMU mapping to PCIe

[renaud]

I have a boot loader to which I am adding LPAE support to make use of the PCIE bus

I have succeed to use a NVME device doing so. However, it fails when I remap the PCIe memory to both bus.

If the code remaps the memory as follows:

VA                           CPU ADDR                     PCIE addr

0x2400.0000 ==> 0x40.0000.0000 ==>    conf space (no device connected there/device node not enabled)

0x3400.0000 ==> 0x48.0000.0000 ==>    conf space (nvme device found on PCIe2)

0x5000.0000 ==> 0x48.4000.0000 ==>   0x4000.0000 mem space on PCIe2

With this MMU configuration, I can see the NVME device and access the device.

If I add the MMU mapping for PCIe1

0x4000.0000 ==> 0x40.4000.0000 == > 0x?000.0000 mem space on PCIe1

then after probing the NVME device on PCIe2,:

nvme pci-126f:2263.0: serial: A012410180629000000
nvme pci-126f:2263.0: model: SM681GEF AGS
nvme pci-126f:2263.0: firmware: TFX7GB

the system hangs.

I do not understand why a MMU mapping to an unused device would hang the system

[Bio_TICFSL]

Hello,

Device tree is board specific. However the LS1021A has fixed memory map and there are 2 address ranges which can be used only for access to PCIe devices. 32 GB at 0x400000_0000 for the PCIe controller1 and 32 GB at 0x480000_0000 for the PCIe controller 2. So it is expected that for the LS1021A based board which use PCIe the device tree programs these 2 outbound regions. Notice both PCIe controllers can operate only as Root Complex. So NIC should be a PCIe End Point. NIC device driver can set inbound window if required. All MSI capable devices implement the MSI capability structure defined in the PCIe Specification. System software is ultimately responsible for enabling MSI support within a device.

Regards

[renaud]

Other thing I found out is that transfer from the NVME device are very slow (1MD/s). After setting the NS bit in the PTE, I got 44MB/s.  

I cannot either boot Linux using an image that used to boot when the boot loader did not use LPAE.