PCIe Endpoint Write to MPC8640D RC's RAM

[ssa]

Hi,
We have a SBC with MPC8640D as root complex(RC) connected to an FPGA endpoint(EP) on PCIe. The FPGA has a DMA module. I am trying to implement a PCIe EP initiated DMA write from FPGA local RAM to MPC8640D's(root complex) RAM.
An rx_buffer of 1MB is defined (as variable in software) in MPC8640D RC RAM at 0x0030 0000 to receive the data.
The EP is enumerated by the RC on power on. The rx_buffer reads value all zeros.
The EP DMA is now configured to write from EP local RAM to EP BAR(in EP's memory map) for upstream PCIe transfer which is set to translate (by EP PCIe core) to the address of rx_buffer in RC(0x0030 0000). The EP shows that DMA transfer is complete. But when I try to read the rx_buffer from MPC8640, the processor crashes.
Am I missing any steps in order to write to MPC8640's RAM from EP? Should I configure PCI Express Inbound Translation Address Registers or PCI Express Inbound Window Base Address Registers?

[ufedor · Accepted solution]

> Should I configure PCI Express Inbound Translation Address Registers or

> PCI Express Inbound Window Base Address Registers?

Yes.

It is required to configure in the MPC8640 RC:

PCI Express Inbound Window Base Address + Base Extended Address

PCI Express Inbound Translation Address

PCI Express Inbound Window Attributes

Refer to the MPC8641D Integrated Host Processor Family Reference Manual, 16.3.5.2.2 RC Inbound ATMU Implementation.

NOTE:
The base address range of all RC’s inbound windows must be configured to reside outside of the Mem_Base & Mem_Limit range  (see RM, 16.3.8.3.11 PCI Express Prefetchable Memory Base Register—0x24 and 16.3.8.3.12 PCI Express Prefetchable Memory Limit Register—0x26).

[ssa]

Thanks @ufedor.

The PCI express inbound window 1 is configured as suggested. The rx_buffer data still read all zeros (processor is not crashing on read). Any hint to troubleshoot the issue?

LAW of PCIe controller 1 is assigned from 0x5000_0000 to 0x5FFF_FFFF. The EP is enumerated and BAR is assigned at 0x5400_0000. The MPC8640 processor test write and read to this EP BAR is successful.

Please correct any error in the configuration mentioned below.
1. PCI Express Inbound Translation Address Register PEXITAR1 is set to the address of rx_buffer[1MB] (0x003c_a000) in DDR where the inbound data has to be finally copied.
2. PCI Express Inbound Window Base Address Register PEXIWBAR1 is set to 0x7000_0000. The EP messages contain the same address so that the window1 will be hit. Note that this address is NOT assigned to any LAWBARs. Should the PEXIWBAR1 fall within LAW of PCIe controller 1 (0x5000_0000 to 0x5FFF_FFFF)?
3. PCI Express Inbound Window Base Extended Address Register PEXIWBEAR1 (supports only 32-bit addressing) does not exist.
4. PCI Express Inbound Window Attributes Register PEXIWAR1 is set - Enabled, Not prefetchable, target interface as Local memory space(for DDR) , Read/Write don’t snoop local processor and size of 1MB.
5. The prefetchable base and limit registers are read as 1, so the inbound window is well outside the prefetchable area.

6. PCI Express Error Detect Register (PEX_ERR_DR) reads 0x0000_0000.

[ufedor]

1) set PEXITAR1=0

2) You wrote:

> PEXIWBAR1 is set to 0x7000_0000. The EP messages contain the same address

Let the EP transaction address be 0x703c_a000

[ssa]

Thanks @ufedor.

It's working with PEXITAR1 is set to the address of rx_buffer[1MB] (0x0_003c_a000). I wrongly configured PEXITAR1 without considering that the platform internal address is 36bits, not 32bits.