topic Re: LS1043A-RDB, outbound PCIe ATU in Layerscape

LS1043A-RDB, outbound PCIe ATU

met — Mon, 13 Feb 2017 13:40:04 GMT

Hi!

I am trying to understand how the outbound PCI address translation is configured in the device tree for the LS1043A-RDB, and how I can modify it to suit our needs.

In the device tree file fsl-ls1043a.dtsi in SDK 2, I found this:

pcie@3400000 {
compatible = "fsl,ls1043a-pcie", "snps,dw-pcie";
...
ranges = <0x81000000 0x0 0x00000000 0x40 0x00010000 0x0 0x00010000 /* downstream I/O */
0x82000000 0x0 0x40000000 0x40 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */

As far as I can tell, this allocates 1 GB worth of address space space for memory.

However, the system memory map shows that a 32 GB address section is set aside for each of the three PCI Express controllers in this device.

Is there a particular reason why so little address space is allocated? Can we increase it from 1 GB?

Also, it is not clear to me why the address ranges are also offset by 1 GB:

- PCI base address: 0x40000000

- System memory base address: 0x40 0x40000000

Is there any reason not to start the address ranges at zero, like below?

- PCI base address: 0x00000000

- System memory base address: 0x40 0x00000000

In our application, we need at least twice as much memory address space, i.e., 2 GB.

I expect to simply have to change the last line in ranges to:

0x82000000 0x0 0x00000000 0x40 0x00000000 0x0 0x80000000>; /* non-prefetchable memory */

This would start the addressing at 0, and doubles the size to 2 GB.

Is this safe to do?

Thank you in advance for any help and insight in this issue!

Re: LS1043A-RDB, outbound PCIe ATU

Pavel — Wed, 15 Feb 2017 05:34:33 GMT

Look at PCI Address Translation on the following page:

http://elinux.org/Device_Tree_Usage

Have a great day,
Pavel Chubakov

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

Re: LS1043A-RDB, outbound PCIe ATU

met — Mon, 20 Feb 2017 15:15:23 GMT

Thanks Pavel,

That guide is informative, but it doesn't really answer much of what I was wondering about.

Allow me to elaborate on my own questions, findings and assumptions:

> Is there a particular reason why so little address space is allocated? Can we increase it from 1 GB?

I assume that the address space is limited to 1 GB simply because no one really put any thought into it.
This is unfortunate, because it is selling the device short. As I understand, it is capable of up to 32 GB address space, and I assume NXP is interested in showing off the device's capabilities.

However, it also seems to me that the PCIe driver itself has a size limit of 4 GB, which stems from the underlying Designware driver because it:
- uses a 32-bit variable for the memory size: drivers/pci/host/pcie-designware.h
- uses only the last mapped memory range from the device tree: drivers/pci/host/pcie-designware.c
This is perhaps a bigger problem, and I urge NXP to fix it.

> Is there any reason not to start the address ranges at zero, like below?

I can only assume the memory region doesn't start at PCI address zero for two reasons:
1) The PCI addresses of I/O and memory regions should not overlap, even though they are different address spaces.
2) The memory region should have a base PCI address which is aligned with the region's size, or potentially the "biggest BAR" in the system.

Note that this means that if the size of the memory region is increased, its PCI base address must also increase.
But you cannot increase the region's size too much because it also reduces the addresses available for any 32-bit PCI devices. With a 4 GB memory range, you could not allocate addresses to _any_ 32-bit PCI devices.

I don't know if point 1) is true. I/O and memory are different spaces after all..

But with all this in mind, perhaps it would make more sense to start the memory region at PCI address 0, and make it _almost_ 4 GB in size?
The I/O region could then occupy the remainder of the 32-bit address space.

I would love input from an expert on this.

Best regards,

Martin Etnestad

Re: LS1043A-RDB, outbound PCIe ATU

Pavel — Thu, 30 Mar 2017 10:03:13 GMT

Sorry for delay.

I do not know a reason for similar setting.

There are no limitation for assigning 1 GB for address space and setting different start address.

Have a great day,
Pavel Chubakov