Enabling PCIe End-point framework on iMX95 torradex board and iMX8MM EVK

1.1 Introduction

PCI Endpoint Framework is a system within the Linux kernel, enabling developers to test the functionality of a PCIe end-point device. Linux kernel simulates a PCIe End-point's behaviour and interact with the PCIe bus. This helps developers to test and validate the PCIe Root Complex providing a structured way to verify the PCIe data transfers.

For detailed info, please go through the official documentation - 9. PCI Endpoint Framework — The Linux Kernel documentation

This article focuses on how to enable the End-point test framework on imx95 and imx8mm.
For the demonstration, iMX95 board will act as a Root Complex and imx8mm will be acting as an End-point.
On the End-point[imx8mm], the framework creates endpoint controller driver, endpoint function driver, and using configfs interface to bind the function driver to the controller driver.

At the RC, a user-space utility called 'pcitest' will be used to read and write data from/to Endpoint.

Enabling this on iMX EVKs is fairly straightforward so chances are less that you will encounter

Issues while enabling this. Nevertheless, if you do face issues in enabling it, please feel free drop in a text so that we can answer your query.

At the end of this exercise, you will be able to send and receive PCIe data from Root Complex[imx95] to End-Point[imx8mm] with/without DMA.

Connections:-

iMX95 Torradex RC will be connected to iMX8MM EVK via M.2 PCIe bridge

iMX95 Torradex board [RC] connected to iMX8MM[EP] via PCIe bridge on M.2

1.2 Changes required on imx95 and imx8mm linux configs:

Kernel configs

# PCI Endpoint

CONFIG_PCI_ENDPOINT=y

CONFIG_PCI_ENDPOINT_CONFIGFS=y

CONFIG_PCI_EPF_TEST=y

1.3 How to run the PCIe end-point test framework?

Build standalone imx8mm and imx95 linux kernel after enabling the linux configs mentioned in the section 1.2
Note- you can also use yocto to build the kernel for imx95 and imx8mm after making the kernel config changes. I use standalone for quick validation and debugging.

After building, you will get kernel images for both imx95 and imx8mm in the location - linux-imx/arch/arm64/boot/Image of the respective imx folders.
Flash imx8mm with an official linux factory image[latest preferred] to emmc

a. Replace the kernel 'Image' of imx8mm [built with end-point configs in the step-1] with the one that comes with the default factory image.

Location of kernel Image on the imx8mm emmc partition - /run/media/boot- mmcblk2p1/Image

Note- If you are using yocto, you can also just flash the built wic image and it will be automatically taken care of[given that the wic was correctly built with the kernel configs mentioned in 1.2]

b. Boot Linux with this dtb -> imx8mm-evk-pcie-ep.dtb

location of the dts in the linux bsp -
linux-imx/arch/arm64/boot/dts/freescale/imx8mm-evk-pcie-ep.dts

If you closely observe this dts:-

It is just disabling the default pcie0 node and enabling the pcie0_ep node. This is because the PCIe drivers need some type of indication from the dtb at boot up so that end-point controller can be created via EPC driver in the Linux kernel.

c. Run the following script to configure the iMX8MM as an endpoint

root@imx8mmevk:~# cat conf_pci_ep

cd /sys/kernel/config/pci_ep/;

mkdir functions/pci_epf_test/func1;

cat functions/pci_epf_test/func1/deviceid;

cat functions/pci_epf_test/func1/vendorid;

echo 0x1957 > functions/pci_epf_test/func1/vendorid;

echo 0x0808 > functions/pci_epf_test/func1/deviceid;

echo 16 > functions/pci_epf_test/func1/msi_interrupts;

echo 8 > functions/pci_epf_test/func1/msix_interrupts;

ln -s functions/pci_epf_test/func1 controllers/33800000.pcie_ep/

root@imx8mmevk:~# ./conf_pci_ep

0xffff

root@imx8mmevk:~#

3. Flash the official imx95 image on the board.

Similar to imx8mm, Boot the imx95 board with the kernel 'Image' built in step-1

In the booting logs, if debugs are enabled, one can observe that the pci_endpoint_test probe will be called. On the linux prompt you can see a device will be created for it.

On lspci output of imx95, you can see the pcie endpoint entry

0808 is the device id we mentioned on the end-point imx8mm in the step-2 above.

4. Now, run the below script on imx95:-

root@imx95-19x19-lpddr5-evk:~# cat pcie_send_to_eptest.sh

#!/bin/sh

# SPDX-License-Identifier: GPL-2.0

echo "PCIe End-point test"

pcitest -r -d -s 102400

pcitest -w -d -s 102400

root@imx95-19x19-lpddr5-evk:~#

The above script will read 102400 bytes from the EP and write 102400 bytes to the EP.

root@imx95-19x19-lpddr5-evk:~# ./pcie_send_to_eptest.sh

PCIe Hot-plug test

[ 2885.375620] pci-endpoint-test 0000:01:00.0: in pci_endpoint_test_ioctl cmd:0x40085005

READ ( 102400 bytes): OKAY

WRITE ( 102400 bytes): OKAY

That's all you need to get started with this simple end-point test framework on imx. Please feel free to ask questions if any.