AnsweredAssumed Answered

Routing PCIe traffic between RCs (PEX)

Question asked by Brett Wilson on Sep 20, 2019
Latest reply on Sep 23, 2019 by Brett Wilson

I've got a board with a T2080 processor on it and it is configured to use 3 PCIe ports, PEX1, PEX2, and PEX4.  Is it possible in HW (no additional SW required, except to configure registers) to route end point traffic between two endpoints that each are on a different PEX and the data does not and should not go into local memory of the T2080.  They would still need to support going into/out of local memory as well on an individual basis and this has been verified to work.

 

Our board vendor pointed to section 2.5.3.1 of the manual and said it is not possible based on what this section said but it's not entirely obvious what this passage is saying and that it does indeed imply that it is not possible.

 

2.5.3.1 Illegal Interaction Between Inbound ATMUs and LAWs
Since both local access windows and inbound ATMUs map transactions to a target
interface, it is essential that they not contradict one another.
For example, it is considered a programming error to have an inbound ATMU map a
transaction target to the local memory space if the resulting translated local address is
mapped to an external peripheral interface by a local access window. Such programming
errors may result in unpredictable system deadlocks.

 

So in our particular example.  One endpoint is on PEX2 and has a BAR address of 0xC030_0000 and the other endpoint is on PEX4 and has a BAR address of 0x8000_0000.  The memory map is as follows:

0x0000_0000 - 0x7FFF_FFFF   SDRAM (2GB)

0x8000_0000 - 0x9FFF_FFFF   PEX4 Memory (512MB)

0xA000_0000 - 0xBFFF_FFFF   PEX1 Memory (512MB)

0xC000_0000 - 0xCFFF_FFFF   PEX2 Memory (256MB)

.... which also includes PCI I/O space of 8MB for each PEX but I'm not concerned about those.

 

If it's possible what do I need to do to enable that transaction between PEX4 and PEX2?  Is there example code?  I'm running VxWorks 6 but if there is even Linux example code that would be helpful.  I've already verified that traffic flows between the T2080 and each end point on each PEX in both directions, just not between PEXs

 

Thank you,

Brett Wilson

Outcomes