i.MX6 PCIe: CFG TLP generation.

[TonyRiederer]

Hello,

We are attempting to use the i.MX6 with a generic PCIe switch. After reviewing the PCIe controller documentation (Chapter 48 of the RM, Rev 0) we have made the following changes to the linux kernel driver:

• We expanded the iATU region 1 mapping from 1M to 2M (1MB per bus)
• We enabled address shift mode in PCIE_PL_iATURC2.
• We are constructing our untranslated addresses using the fields specified in Table 48-23.  This required changing the macros (pcie.c) which did not match either mapping defined in the RM. The existing code works with a single device because the BDF value is always 0.

With or without these changes, we are able to access the CFG space of the upstream switch port (Bus 1, BDF = 0.0.0 )  including the PCIe specific registers. However, we cannot successfully read the CFG space for the downstream ports associated with bus 2 (BDF = 1.0.0).

The behavior we are seeing is:

• Changing the Bus and Device fields in the untranslated address has no discernible effect on the CFG read/write cycle. We always get back the Bus 1 Device 0 values regardless of the D27:15 values.
• Changing the function number to non-zero causes an abort. This is not expected as the RM states that the function number in the outbound TLP is derived from PCIE_PL_iATURC1 D22:20
• Accessing bus 1 (BDF = 0.0.0) registers >= 0x400 causes an abort.

Any clarification on how to configure the imx6 to generate the proper CFG TLP would be greatly appreciated.

Thanks,

Tony

[nilsb · Accepted solution]

PCI-E BSP file with support for multiple levels of PCI busses (e.g. PCI-PCI bridges)

arch/arm/mach-mx6/pcie.c

[TonyRiederer]

Hi Nils,

That fixed the Config1 TLP generation issue. Many thanks!

The next problem is that the (linux) enumeration crashes whenever a UR status is returned from a downstream device. The first downstream port on our switch is device.1 (not 0). We've hacked around this for the time being by hard coding some BDF limits in the rd/wr functions. I'll post  the code when we've implemented the proper abort handling. How are you getting around this in your design?

Tony

[nilsb]

Our PCI device is device 0 and only supports function 0, so I have not tested for non-zero devfn values. Are you seeing 'imprecise external abort' panics?  This is what I saw during probing for non-zero devfn values, so I hardwired the code to return failure (0xffffffff) for PCI config read/writes to devfn!=0, mimicking the root port case (again in pcie.c).

To trap the error, one would trap the bus error through the page fault handler (arch/arm/mm/fault.c), right?   Use something like hook_fault_code()? (I'm asking for my own edification)

[EricNelson]

Hi Tony and Nils,

This patch on our pcie-patches-01.01 kernel branch was needed for us to get a PLX PCIe bridge to function.

[TonyRiederer]

Nils,

That seems to be a comon approach in other ARM based PCI implementations. It would be nice if there was a cleaner way to handle an error, but there's nothing obvious in the Freescale RM.

"Imprecise data aborts

The state of the system presented to the abort exception handler for an imprecise data abort can be the state for an instruction after the instruction that caused the abort. As a result, it is not often possible to restart the processor from the point where the exception occurred."  - Cortex-A8 Technical Reference Manual, 2.15.6

In our case, the exception address presented to the fault handler is +2 instructions from the one which caused the abort.

Tony

[nilsb]

I recently solved what I believe is a similar if not identical problem. The BSP is not compliant with the RM, as you noticed with respect to issuing type 1 PCI configuration transactions. Here's is how I approached it.

Please see arch/arm/mach-mx6/pcie.c

- Do not use the address shift mode. Use the approach in RM Sec. 48.3.9.2 (rev 0) i.e. PCI bus # is shifted 24 bits.

- The memory mapped register address should be constructed as:

     va_address = (u32)base + PCIE_CONF_REG(where);

- Use the iATU to set the appropriate BDF bits (via the target address register (low 32-bits)), i.e.:

atu_bdf_addr = PCIE_CONF_BUS(bus->number) | PCIE_CONF_DEVFN(devfn);

writel(atu_bdf_addr, dbi_base + ATU_REGION_LOW_TRGT_ADDR_R);

-If the bus number is greater than 1: Since PCI-bridges are being traversed, ATU must translate this request to Type-1 Config transaction.

writel(CfgRdWr1, dbi_base + ATU_REGION_CTRL1_R);

- If the bus number is 1 or less: You need to use a type 0 config transaction.

writel(CfgRdWr0, dbi_base + ATU_REGION_CTRL1_R);

I.e. you are updating the iATU configuration for each transaction. I have not included all the changes I had to make, these are just the main ideas. I am not sure how to submit the entire file to this forum, but that would obviously be easier.

Hope this helps,

Nils

[TonyRiederer]

Hi Nils,

Thanks for the information.

If you select "Use advanced editor" when writing a reply you can attach files using a button that appears at the lower left.

Tony