QDMA Data Transfer between PCIe Endpoint and DDR Memory on LX2160A Platform

[#1]

Introduction

The qDMA controller transfers blocks of data between one source and one destination. The blocks of data transferred can be represented in memory as contiguous or noncontiguous using scatter/gather table(s). Channel virtualization is supported through enqueuing of DMA jobs to, or dequeuing DMA jobs from, different work queues. QDMA can support LX2160A with DPAA2.

The attached patch is used for QDMA data transfer between PCIe Endpoint and DDR memory on LX2160A Platform.

Apply the patch and rebuild Kernel image in LSDK 21.08

Go to LSDK 21.08 Linux Kernel source folder components/linux/linux/

Apply the attached patch.

$ patch -p1 < 0001-QDMA-Performance-test.patch

$ flex-builder -c linux:custom -a arm64

Enable CONFIG_FSL_DPAA2_QDMA=y

$ Rebuild Linux Kernel.

$ flex-builder -c linux -a arm64

 

Transfer data from Memory to Memory

root@localhost:~# restool dpdmai create --num-queues=8 --priorities=2,5

dpdmai.0 is created under dprc.1

root@localhost:~# restool dprc assign dprc.1 --object=dpdmai.0 --plugged=1

root@localhost:~# echo "dpdmai.0" > /sys/module/dmatest/parameters/device

root@localhost:~# echo '7       4       1       7' > /proc/sys/kernel/printk

root@localhost:~# echo 1 > /sys/module/dmatest/parameters/dmatest_option

root@localhost:~# echo 10 > /sys/module/dmatest/parameters/iterations

root@localhost:~# echo 4096 > /sys/module/dmatest/parameters/test_buf_size

root@localhost:~# echo 1 > /sys/module/dmatest/parameters/threads_per_chan

root@localhost:~# echo 1 > /sys/module/dmatest/parameters/run

[  177.903819] dmatest: current_mask 1

root@localhost:~# [  177.907636] dmatest: len 11         save_mask ffff  tmp_mask 2

[  178.927396] dmatest: qdma-test-task: summary 10 tests, 0 failures 303030 iops 1212121 KB/s 1241212121 B/s (0)

[  178.937308] dmatest: total_len 40960  len 4096

[  178.941870] dmatest: runtime 33

[  178.941870]

root@localhost:~# echo 8 > /sys/module/dmatest/parameters/threads_per_chan

root@localhost:~# echo 1 > /sys/module/dmatest/parameters/run

[  273.731698] dmatest: current_mask 1

root@localhost:~# [  273.735625] dmatest: len 11         save_mask ffff  tmp_mask 2

[  274.767409] dmatest: qdma-test-task: summary 10 tests, 0 failures 357142 iops 1428571 KB/s 1462857142 B/s (0)

[  274.777315] dmatest: total_len 40960  len 4096

[  274.781875] dmatest: runtime 28

[  274.781875]

Transfer data from Memory to PCIe 

root@lx2160ardb-rev2:~# restool dpdmai create --num-queues=8 --priorities=2,5

dpdmai.0 is created under dprc.1

root@lx2160ardb-rev2:~# restool dprc assign dprc.1 --object=dpdmai.0 --plugged=1

root@lx2160ardb-rev2:~# echo "dpdmai.0" > /sys/module/dmatest/parameters/device

root@lx2160ardb-rev2:~# echo '7       4       1       7' > /proc/sys/kernel/printk

root@lx2160ardb-rev2:~# echo 4096 > /sys/module/dmatest/parameters/test_buf_size

root@lx2160ardb-rev2:~#  echo 10 > /sys/module/dmatest/parameters/iterations

root@lx2160ardb-rev2:~# echo 2 > /sys/module/dmatest/parameters/dmatest_option

root@lx2160ardb-rev2:~# echo 0x9040000000 > /sys/module/dmatest/parameters/pci_phys_addr

root@lx2160ardb-rev2:~# echo 1 > /sys/module/dmatest/parameters/produ_task_number

root@lx2160ardb-rev2:~# echo 1 > /sys/module/dmatest/parameters/run

[  214.319248] dmatest: current_mask 1

root@lx2160ardb-rev2:~# [  214.322864] dmatest: len 11   save_mask ffff  tmp_mask 2

[  214.330187] dmatest: pcie_phy_base : 9040000000       pcie_virt_base : 0000000014415a49

[  215.355448] dmatest: qdma-test-task: summary 10 tests, 0 failures 26737 iops 106951 KB/s 109518716 B/s (0)

[  215.365093] dmatest: total_len 40960  len 4096

[  215.369529] dmatest: runtime 374

[  215.369529]