Issue with i.MX6SoloX as PCIe EP

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Issue with i.MX6SoloX as PCIe EP

952 Views
keestrommel
Contributor IV

I started with reproducing the i.MX6Q PCIe EP/RC validation done by NXP using iMX6Q and i.MX6SoloX development board. At both boards Linux imx-4.9.11-1.0.0_ga SDK is used.

The i.MX6SoloX is configured as RC and the i.MX6Q as EP.  The test run automatically by the EP is successful and produces the following result:

Oct 19 01:51:26 imx6qsabresd user.info kernel: pcie ep: Starting data transfer...
Oct 19 01:51:26 imx6qsabresd user.info kernel: pcie ep: Data transfer is successful. tv_count1 34682us, tv_count2 104120us.
Oct 19 01:51:26 imx6qsabresd user.info kernel: pcie ep: Data write speed:59MB/s.
Oct 19 01:51:26 imx6qsabresd user.info kernel: pcie ep: Data read speed:19MB/s.

lspci at the RC is also as expected:

root@imx6sxsabresd:~# lspci -v
00:00.0 PCI bridge: Synopsys, Inc. Device abcd (rev 01) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 309
        Memory at 08800000 (32-bit, non-prefetchable) [size=1M]
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00001fff
        Memory behind bridge: 08000000-087fffff
        Prefetchable memory behind bridge: 08900000-089fffff
        [virtual] Expansion ROM at 08a00000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Capabilities: [70] Express Root Port (Slot-), MSI 00
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: pcieport

01:00.0 RAM memory: Device beaf:dead (rev 01)
        Subsystem: Device beaf:dead
        Flags: bus master, fast devsel, latency 0, IRQ 310
        Memory at 08000000 (32-bit, non-prefetchable) [size=8M]
        I/O ports at 1000 [size=4K]
        Memory at 08910000 (32-bit, prefetchable) [size=256]
        [virtual] Expansion ROM at 08900000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Kernel driver in use: imx_pcie_ep

So far, so good. My next step is to access the EP RAM from the RC. Because the outbound iATU at the RC added by the NXP PCIe EP compliance test code in drivers/pci/host/pci-imx6.c did work did not work this outbound iATU has been removed from the RC. Instead using memtool like commands the inbound iATU has been added at the EP:

root@imx6qsabresd:~# rio 32 1ffc900 20

<0x1ffc900> 0x80000000 0x00000000 0xc0000000 0x00000000
<0x1ffc910> 0x00000000 0x0000ffff 0x4f000000 0x00000000

This is an inbound iATU that translates the BAR0 region to the EPs test RAM region. This works fine, the RAM at the EP can be accessed using the PCIe memory region at the RC.

RC:

root@imx6sxsabresd:~# wio 32 8000000 deadbeef
<0x8000000> 0xdeadbeef
root@imx6sxsabresd:~# rio 32 8000000 20
<0x8000000> 0xdeadbeef 0xf79ef79e 0xf79ef79e 0xf79ef79e
<0x8000010> 0xf79ef79e 0xf79ef79e 0xf79ef79e 0xf79ef79e

EP:

root@imx6qsabresd:~# rio 32 4f000000 20
<0x4f000000> 0xdeadbeef 0xf79ef79e 0xf79ef79e 0xf79ef79e
<0x4f000010> 0xf79ef79e 0xf79ef79e 0xf79ef79e 0xf79ef79e

Note: Because the kernel did not start with the "mem=768M" therefore "mem=1008" is used and the ddr_test_region is moved 0x4f000000 (i.MX6Q) or 0xbf000000 (i.MX6SoloX).

For our purpose we want to access the EP peripherals (e.g. USB) from the RC, so I changed the target address of the inbound iATU to the AIPS (0x2000000)

EP:

root@imx6qsabresd:~# rio 32 1ffc900 20
<0x1ffc900> 0x80000000 0x00000000 0xc0000000 0x00000000
<0x1ffc910> 0x00000000 0x0000ffff 0x02000000 0x00000000
root@imx6qsabresd:~# rio 32 2184200 18
<0x2184200> 0xe401fa05 0x00000115 0x10020001 0x00000000
<0x2184210> 0x80080808 0x00000808

RC:

root@imx6sxsabresd:~# rio 32 8184200 18
<0x8184200> 0xe401fa05 0x00000115 0x10020001 0x00000000
<0x8184210> 0x80080808 0x00000808

So when the i.MX6SoloX is the RC and the i.MX6Q the EP everything is working as expected, but when the EP and RC roles are swapped it is possible to access the EPs RAM from the RC but its is not possible to access the EPs peripherals.

EP (set inbound iATU to test RAM and read test RAM):

root@imx6sxsabresd:~# rio 32 8ffc900 20
<0x8ffc900> 0x80000000 0x00000000 0xc0000000 0x00000000
<0x8ffc910> 0x00000000 0x0000ffff 0xbf000000 0x00000000
root@imx6sxsabresd:~# rio 32 bf184200 18
<0xbf184200> 0x00000000 0x00000000 0x00000000 0x00000000
<0xbf184210> 0x00000000 0x00000000

RC (write and read back EPs test RAM):

root@imx6qsabresd:~# wio 32 1184200 deadbeef
<0x1184200> 0xdeadbeef
root@imx6qsabresd:~# rio 32 1184200 18
<0x1184200> 0xdeadbeef 0x00000000 0x00000000 0x00000000
<0x1184210> 0x00000000 0x00000000

EP (set inbound iATU to AIPS and read USB registers):

root@imx6sxsabresd:~# wio 32 8ffc918 2000000
<0x8ffc918> 0x2000000
root@imx6sxsabresd:~# rio 32 8ffc900 20
<0x8ffc900> 0x80000000 0x00000000 0xc0000000 0x00000000
<0x8ffc910> 0x00000000 0x0000ffff 0x02000000 0x00000000
root@imx6sxsabresd:~# rio 32 2184200 18
<0x2184200> 0xe401fa05 0x00000015 0x10020001 0x00000011
<0x2184210> 0x80080b08 0x00000808


RC (read EPs USB registers):

root@imx6qsabresd:~# rio 32 1184200 18
Unhandled fault: external abort on non-linefetch (0x1018) at 0x76f47200
pgd = a2724000
[76f47200] *pgd=32875831, *pte=01184703, *ppte=01184e33
Bus error (core dumped)


Did I miss something when using the i.MC6SoloX as EP or is this an iMX6SoloX issue?

Labels (1)
0 Kudos
2 Replies

602 Views
Yuri
NXP Employee
NXP Employee

Hello,

  in the original thread https://community.nxp.com/docs/DOC-95014 

the follwoing is mentioned "In EP's system, EP can access the reserved ddr memory  ..."

and the patch and kernel line settings are shown. Have You performed it ?

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos

602 Views
keestrommel
Contributor IV

In the latest linux-imx (4.9.11-1.0.0_ga) the patches you mentioned are already merged to main. So I did not need to apply the patches. Anyhow as described I could access the other boards RAM both from the the EP and RC. The issue is when I try to access the USB registers. This does not work if the i.MX6SoloX is the EP and the i.MX6Q is the RC. If the EP and RC roles are swapped it is possible.

0 Kudos