[iMX8MM PCIe] What is the position of the CS2 address bit in endpoint mode?

[amartin_witekio]

Hello,

  First off, my actual problem:

When using Linux's pci-epf-test driver on an iMX8MM, I can't seem to collect the right BARs configuration from the host side (also Linux on an x86, I tried several hardwares in 32 and 64 bits).

Actually, my host pci_endpoint_test module gets the respective flags:

• BAR0: 0x00040200 (IORESOURCE_MEM | IORESOURCE_SIZEALIGN)
• All other BARs: 0x00000000

On the endpoint side, the test driver tries to allocate BARs 0, 2, 4 and 5, avoiding the ones declared reserved by the iMX driver.

As much of the reconnoissance by the host is performed by its BIOS, I don't have much visibility there, so I tried to follow the BAR configuration code on the endpoint to check everything was right on that side.

Such configuration is performed by the more generic Designware layer. It consists of 2 parts:

• First the configuration of an inbound region in the iATU. As far as I can see, the iATU registers are configured as they should, as confirmed by reading back their values faterwards.
• Then the BAR and BAR_MASK registers, and that's where things get ugly.

It gets ugly for 2 reasons:

• Both those registers, are write-only, which means I have no way of verifying their good programmation until the process works end-to-end.
• Also, the BAR_MASK registers are accessed through DBI2, "that is,
  assert the dbi_cs2 input, or the CS2 address bit for the AXI bridge".

Wonderful, we access DBI through the AXI bridge, so remains only to locate the CS2 address bit.

It appears the imx6-pci driver uses bit 20 for iMX8 MM, MP and MQ, and bit 12 for all other families.

Indeed, the iMX6DQ RM mentions once in passing: "The BAR Mask registers are accessible through the same address as the corresponding BAR registers, but requires dbi_cs2 assertions instead i.e. bit address 12 must be set."

For iMX8MM no such precision, but the iMX8DQXP RM gives us something a bit more... complex, through a handful of tables. Page 8392 we find:

• If CX_ARI_ENABLE = 0 and CX_SRIOV_ENABLE = 0, CS2 lives at position 20.
• If CX_ARI_ENABLE = 1 or CX_SRIOV_ENABLE = 1, CS2 is bit 31.

But on the next page: "Bit position is based on CX_NFUNC and not on device_type", and

• CX_NFUNC =1; CX_ARI_ENABLE=0: CX_SRIOV_ENABLE=0 => bit 12
• CX_NFUNC >1; CX_SRIOV_ENABLE=0; CX_ARI_ENABLE=0) => bit 19
• CX_NFUNCa >1 and CX_ARI_ENABLE=0 and CX_SRIOV_ENABLE=0 and DBI_MULTI_FUNC_BAR_EN=1 => bit 20
• in EP Mode (CX_SRIOV_ENABLE=1 or CX_ARI_ENABLE=1) => bit 31

But I have no idea as to how these internal signals are set. Am I to understand that my using PCIe as endpoint implies [CX_SRIOV_ENABLE=1 or CX_ARI_ENABLE=1]?
If so, how am I to refer to the resulting addresses, when they fall into RAM address space?
And even then, the base offset used by the Designware layer to address register BARx on db1 (respectively BARx_MASK on dbi2) is 0x10+4*x. But those addresses are either undocumented (0x10, 0x14) or used for something else entirely (and writable or becoming so on the same condition as the BAR registers, by setting DBI_RO_WR_EN in MISC_CONTROL_1):

• 0x18: Secondary Latency Timer, Subordinate Bus Number, Secondary Bus Number, and Primary Bus Number Register
• 0x1c: Secondary Status, and I/O Limit and Base Register
• 0x20: Memory Limit and Base Register
• 0x24: Prefetchable Memory Limit and Base Register

So, I'm lost. How am I to access the BAR registers?

Best regards

[jimmychan]

Could you tell me when version of BSP are you using?

[amartin_witekio]

We're using Yocto Zeus with a kernel based on rel_imx_5.4.70_2.3.2 from CodeAurora.

However, though my post presents observations made with that BSP and read from this kernel code, the question itself is solely on the hardware implementation of the PCIe IP for iMX8MM.