I have 2 T2080s attached to each other on PCIe. I have 1 setup as the RC and the other setup as the EP. I am only testing traffic (reads/writes) to the EP from the RC. When I run the test for connections between PCIe2 or PCIe4 the test functions correctly and data written to the EP is echoed back correctly on reads from the EP. The test is simply mapping to DRAM on the EP's T2080. So the ATMU registers, LAWs, TLBs are set up correctly.
However, the test fails when using PCIe1. The configuration is exactly the same for all three controller setups. The SW simply is compiled to change the bus from PCIe4 to PCIe1 with appropriate memory changes to the controllers. We have a PCIe analyzer on the bus. We see the writes and reads complete as expected. But the data read back does not represent the data written in the prior step by the RC. When we break in with an emulator from the RC and examine the memory space mapped to the EP DRAM (BAR1) through the PCIe bus, it appears to be a repeating pattern. When I break in to the T2080 EP processor and examine DRAM it is unchanged from the original values written during DRAM init. What is different about PCIe1. I realize that PCIe1 supports SR-IOV, but I do not have that enabled.