How can someone configure the related board as end point device so that one can access board's RAM over PCIe? I have checked the following docs; however, I could not find a solution. Can someone provide an example? or suggest any other doc? Thanks in advance
You could refer to section "7.11.6 PCIE EP" in QORIQ-SDK-2.0-IC-REV0.pdf.
Please refer to LS1046A RDB in PCIe-Endpoint Mode .
1. Modify the rcw for EP board:
modify rcw for RC board as follow to set the linkup speed to 5.0G
Then rebuild the two rcw,
Program the rcw into EP and RC board
Build the image for EP board
$ bitbake -c menuconfig virtual/kernel
-> Device Drivers
-><*>VFIO Non-Privileged userspace driver framework
-><*> VFIO support for Freescale PCI Endpoint devices
-><*> Userspace I/O drivers
<*> Freescale DMA support
3. Boot up the EP board with kerenl rootfs
Deploy the EP board
Run PCI DMA EP application to initialize EP device.
root@p5020ds:~# ./pciep_dma 0
pcidma> add 0 0
4. Build the PCIe RC driver
-> Device Drivers
-> DMA Engine support
[*] Freescale Elo and Elo Plus DMA support (enable the option)
--> [*] PCI IOV support
On the RC board:
Test RC to EP DMA performanceroot@t4240qds:/sys/class# cd pcidma/pcidma0/ root@t4240qds:/sys/class/pcidma/pcidma0# ls bars_info config_infodevicesubsystem test_dma_enable test_ep2rctest_info test_lens test_loop test_rc2ep test_start uevent root@t4240qds:/sys/class/pcidma/pcidma0# echo 1 > test_start test starting root@t4240qds:/sys/class/pcidma/pcidma0# test info: test0 packet length:64B loop:500times RC->EP throughput:49Mbps test1 packet length:256B loop:500times RC->EP throughput:164Mbps test2 packet length:1024B loop:500times RC->EP throughput:658Mbps test3 packet length:4096B loop:500times RC->EP throughput:1594Mbps test4 packet length:1048576B loop:500times RC->EP throughput:11216Mbps test5 packet length:2097152B loop:500times RC->EP throughput:11337Mbps 6.
Test EP to RC DMA performanceroot@t4240qds:/sys/class/pcidma/pcidma0# echo 1 > test_ep2rc root@t4240qds:/sys/class/pcidma/pcidma0# echo 0 > test_rc2ep root@t4240qds:/sys/class/pcidma/pcidma0# echo 1 > test_start test starting root@t4240qds:/sys/class/pcidma/pcidma0# test info: test0 packet length:64B loop:500times EP->RC throughput:353Mbps test1 packet length:256B loop:500times EP->RC throughput:1270Mbps test2 packet length:1024B loop:500times EP->RC throughput:3893Mbps test3 packet length:4096B loop:500times EP->RC throughput:7710Mbps test4 packet length:1048576B loop:500times EP->RC throughput:12148Mbps test5 packet length:2097152B loop:500times EP->RC throughput:12162Mbps
Sorry for the late reply, and thank you for the quick one. We tried your suggestion, but after setting RCWs for EP and RC boards, we experienced some power issues. Can you help us about the following issues? Or can you guide us?
I confirmed with LSDK development team, please refer to the following.
[Yiping] Yes, we use male-to-male PCIe cable connect RC and EP boards, PCIe switch is not necessary.
[Yiping] We’d never try this way.
Please refer to OSS-KIT-EXP-3500 for PCIe Gen 2 x4 https://www.onestopsystems.com/product/pcie-x4-expansion-kit-3500
OSS-KIT-EXP-8000 for PCIe Gen 2 x8 https://www.onestopsystems.com/product/pcie-x8-gen-2-host-target-kit-8000