64 bit and 32 bit PCIe end point config issue in T1042 controller

[yashwantdixit]

we are using freescale T1042 Processor  and in PCIe conroller 2 there are two devices exists with one being 64 bit and other is 32 bit devices. I need to configure it in DTS using ranges field. I have tried following :

	pci1: pcie@ffb250000 {
	reg = <0xf 0xfb250000 0 0x10000>;
	ranges = <0x03000000 0x4 0x00000000 0x4 0x00000000 0x0 0x10000000
	0x02000000 0x0 0x80000000 0x0 0x80000000 0x0 0x10000000>;

in above I have defined two ranges one being 64 bit and other is 32 bit. but this is not working : and I am getting following issue :

PCI host bridge /pcie@ffb250000  ranges:

MEM 0x0000000400000000..0x000000040fffffff -> 0x0000000000000000

MEM 0x0000000080000000..0x000000008fffffff -> 0x0000000080000000

Ran out of outbound PCI ATMUs for resource 0!

/pcie@ffb250000: PCICSRBAR @ 0x7f000000

/pcie@ffb250000: Setup 64-bit PCI DMA window

/pcie@ffb250000: DMA window size is 0x7f000000

can someone please throw some light as how can I solve this issue ?

warm regards,

Yashwant

[scottwood]

It looks like you're trying to set both windows to be 256 MiB -- why not just have one 32-bit 512 MiB window?  A 64-bit device shouldn't require a 64-bit window, unless the required window is too big for 32-bit.

I don't know why you're getting "MEM 0x0000000400000000..0x000000040fffffff -> 0x0000000000000000" rather than "MEM 0x0000000400000000..0x000000040fffffff -> 0x0000000400000000", but I found the cause of the "Ran out of outbound PCI ATMUs" message.  In setup_one_atmu() in arch/powerpc/sysdev/fsl_pci.c, change "__ffs" to "__builtin_ffsll".  This bug only triggers on a 32-bit kernel when the PCI and physical addresses are both zero in the low 32 bits.

[bernhardkaindl]

Scott, __builtin_ffsll is wrong here (__ffs is offset by 1) and it does not build in a 32-bit kernel. For a proper solution that works 32-bit and 64-bit and uses established kernel functions use this version with __ffs64. You can used this patch with with my Signed-off-by line:

This patch should be split into two patches to conform to Documentation/Submitting patches, but should be ok for early review:

Subject: [powerpc-fsl] Fix ATMU configuration in 32-bit kernel when the PCI and physical addresses are both zero in the low 32 bits.

It also checks if physical(cpu) and PCI (bus) addresses are correctly size-aligned.

While the code tries to cope with unaligned addresses to the fullest extent possible, but it only has 4 ATMUs available among which it can split the non size-aligned range into smaller size-aligned ATMUs, so it will run out of ATMUs quickly.

More severely, a mis-algined phys/cpu address likely means that the LAW defined by U-Boot is also not size-aligned, which is fatal since the SoC effectively cuts of the unaligned bits from the start address and the LAW then ends too early, leading to a machine check on each attempt to access the PCI mem region that the LAW cannot correctly cover due to the misalignment.

Signed-off-by: Bernhard Kaindl <berhard.kaindl@thalesgroup.com>

--- source/arch/powerpc/sysdev/fsl_pci.c

+++ source/arch/powerpc/sysdev/fsl_pci.c

@@ -145,19 +145,31 @@

        u32 flags = 0x80044000; /* enable & mem R/W */

        unsigned int i;

-       pr_debug("PCI MEM resource start 0x%016llx, size 0x%016llx.\n",

-               (u64)res->start, (u64)size);

-

        if (res->flags & IORESOURCE_PREFETCH)

                flags |= 0x10000000; /* enable relaxed ordering */

+       if (ilog2(size) > __ffs64(pci_addr | phys_addr)) {

+               resource_size_t bus_align = 1 << (__ffs64(pci_addr));

+               resource_size_t cpu_align = 1 << (__ffs64(phys_addr));

+

+               pr_warn("%pr not size-aligned:\n", res);

+               if (cpu_align < size)

+                       pr_warn("cpu alignment %pa < %pa\n", &cpu_align, &size);

+               if (bus_align < size)

+                       pr_warn("bus alignment %pa < %pa\n", &bus_align, &size);

+       }

+

        for (i = 0; size > 0; i++) {

-               unsigned int bits = min(ilog2(size),

-                                       __ffs(pci_addr | phys_addr));

+               unsigned int bits = min((int)ilog2(size),

+                                       (int)__ffs64(pci_addr | phys_addr));

                if (index + i >= 5)

                        return -1;

+               pr_debug("ATMU: %016llx..%016llx -> %llx\n",

+                       (u64)phys_addr,

+                       (u64)phys_addr + (1 << bits) - 1, pci_addr);

+

                out_be32(&pci->pow[index + i].potar, pci_addr >> 12);

                out_be32(&pci->pow[index + i].potear, (u64)pci_addr >> 44);

                out_be32(&pci->pow[index + i].powbar, phys_addr >> 12);