i.MX6Q PCIe EP/RC Validation System

Document created by Richard Zhu Employee on Jun 20, 2013Last modified by Richard Zhu Employee on Mar 5, 2014
Version 13Show Document
  • View in full screen mode

i.MX6Q PCIe EP/RC Validation and Throughput

Hardware setup

    * Two i.MX6Q SD boards, one is used as PCIe RC; the other one is used as PCIe EP. Connected by 2*mini_PCIe to standard_PCIe  adaptors, 2*PEX cable adaptors,  and one PCIe cable.

2013-03-21_10-24-40_704.jpg

 

Software configurations

    * When building RC image, make sure that

        CONFIG_IMX_PCIE=y

        # CONFIG_IMX_PCIE_EP_MODE_IN_EP_RC_SYS is not set

        CONFIG_IMX_PCIE_RC_MODE_IN_EP_RC_SYS=y

    * When build EP image, make sure that

        CONFIG_IMX_PCIE=y

        CONFIG_IMX_PCIE_EP_MODE_IN_EP_RC_SYS=y

        # CONFIG_IMX_PCIE_RC_MODE_IN_EP_RC_SYS is not set

Features

    * Set-up link between RC and EP by their stand-alone 125MHz running internally.

* In EP's system, EP can access the reserved ddr memory    (default address:0x40000000) of PCIe RC's system, by the   interconnection between PCIe EP and PCIe RC.

NOTE: The layout of the 1G DDR memory on SD boards is 0x1000_0000 ~ 0x4FFF_FFFF)

Use mem=768M in the kernel command line to reserve the 0x4000_0000 ~ 0x4FFF_FFFF DDR memory  space used to do the EP access tests.

(The example of the RC’s cmd-line: Kernel command line: noinitrd console=ttymxc0,115200 mem=768M root=/dev/nfs nfsroot=10.192.225.216:/home/r65037/nfs/rootfs_mx5x_10.11,v3,tcp ip=dhcp rw)

Throughput results

 

ARM core used as the bus master, and cache is disabledARM core used as the bus master, and cache is enabledIPU used as the bus master(DMA)
Data size in one write tlp8 bytes32 bytes64 bytes
Write speed~109MB/s~298MB/s~344MB/s
Data size in one read tlp32 bytes64 bytes64 bytes
Read speed~29MB/s~100MB/s~211MB/s

 

IPU used as the bus master(DMA)

Here is the summary of the PCIe throughput results tested by IPU.

Write speed is about 344 MB/s.

Read speed is about 211MB/s

ARM core used as the bus master (define EP_SELF_IO_TEST in pcie.c driver)

write speed ~300MB/s.

read speed ~100MB/s.

Cache is enabled.

PCIe EP: Starting data transfer...

PCIe EP: Data transfer is successful, tv_count1 54840us, tv_count2 162814us.

PCIe EP: Data write speed is 298MB/s.

PCIe EP: Data read speed is 100MB/s.

Regarding to the log, the data size of each TLP when cache is enabled, is about 4 times of the data size in write, and 2 times of the data size in read, when the cache is not enabled.

 

Cache is disabledCache is enabled
Data size in one write tlp8 bytes32 bytes
Write speed~109MB/s~298MB/s
Data size in one read tlp32 bytes64 bytes
Read speed~29MB/s~100MB/s

 

Cache is not enabled

PCIe EP: Starting data transfer...

PCIe EP: Data transfer is successful, tv_count1 149616us, tv_count2 552099us.

PCIe EP: Data write speed is 109MB/s.

PCIe EP: Data read speed is 29MB/s.

One simple method used to connect the imx6 pcie ep and rc

View of the whole solution:

2013-04-07_15-43-46_818_small.jpg

HW materials:

2* iMX6Q SD boards,  2* Mini PCIe to STD PCIe adaptors, one SATA2 data cable.

the mini-pcie to standard pcie exchange adaptor.

Here is the URL:
http://www.bplus.com.tw/Adapter/PM2C.html

PM2C.png

How to make it.

signals connections

Two adaptors, one is named as A, the other one is named as B.

A                  B

TXM <----> RXM

TXN <----> RXN

RXM <----> TXM

RXN <----> TXN

A1 connected to B3

A2 connected to B4

A3 connected to B1

A4 connected to B2

2013-04-07_16-02-59_594_small.jpg

Connect the cable to the adaptor.

Connect the SATA2 data cable to Mini PCIe to STD PCIe adaptor (A)

2013-04-07_16-02-17_913_small.jpg

 

Connect the SATA2 data cable to Mini PCIe to STD PCIe adaptor (B)

2013-04-07_16-01-36_658_small.jpg

NOTE:
* Please keep length of Cable as short as possible.  Our cable is about 12cm.

* Please connect shield wire in SATA2 Cable to GND at both board.
* Please boot up PCIe EP system before booting PCIe RC system.

Base one imx_3.0.35 mainline, the patch, and the IPU test tools had been attached.

NOTE:

* IPU tests usage howto.

Unzip the xxx.zip, and run xxx_r.sh to do read tests, run xxx_w.sh to do the write tests.

Tests log:
EP:

root@freescale ~/pcie_ep_io_test$ ./pcie-r.sh
pass cmdline 14, ./pcie_ipudev_test.out

new option : c
frame count set 1

new option : l
loop count set 1

new option : i
input w=1024,h=1024,fucc=RGB4,cpx=0,cpy=0,cpw=0,cph=0,de=0,dm=0

new option : O
640,480,RGB4,0,0,0,0,0
new option : s
show to fb 0

new option : f
output file name ipu1-1st-ovfb

new option : ÿ
show_to_buf:0, input_paddr:0x1000000, output.paddr0x18800000
====== ipu task ======
input:
        foramt: 0x34424752
        width: 1024
        height: 1024
        crop.w = 1024
        crop.h = 1024
        crop.pos.x = 0
        crop.pos.y = 0
output:
        foramt: 0x34424752
        width: 640
        height: 480
        roate: 0
        crop.w = 640
        crop.h = 480
        crop.pos.x = 0
        crop.pos.y = 0
total frame count 1 avg frame time 19019 us, fps 52.579000
root@freescale ~/pcie_ep_io_test$ ./pcie-w.sh
pass cmdline 14, ./pcie_ipudev_test.out

new option : c
frame count set 1

new option : l
loop count set 1

new option : i
input w=640,h=480,fucc=RGB4,cpx=0,cpy=0,cpw=0,cph=0,de=0,dm=0

new option : O
1024,1024,RGB4,0,0,0,0,0
new option : s
show to fb 1

new option : f
output file name ipu1-1st-ovfb

new option : ÿ
show_to_buf:1, input_paddr:0x18a00000, output.paddr0x1000000
====== ipu task ======
input:
        foramt: 0x34424752
        width: 640
        height: 480
        crop.w = 640
        crop.h = 480
        crop.pos.x = 0
        crop.pos.y = 0
output:
        foramt: 0x34424752
        width: 1024
        height: 1024
        roate: 0
        crop.w = 1024
        crop.h = 1024
        crop.pos.x = 0
        crop.pos.y = 0
total frame count 1 avg frame time 11751 us, fps 85.099140

root@freescale ~$ ./memtool -32 01000000=deadbeaf
Writing 32-bit value 0xDEADBEAF to address 0x01000000

RC:

Before run "./memtool -32 01000000=deadbeaf" at EP.

root@freescale ~$ ./memtool -32 40000000 10
Reading 0x10 count starting at address 0x40000000

0x40000000:  00000000 00000000 00000000 00000000
0x40000010:  00000000 00000000 00000000 00000000
0x40000020:  00000000 00000000 00000000 00000000
0x40000030:  00000000 00000000 00000000 00000000

After run "./memtool -32 01000000=deadbeaf" at EP.

root@freescale ~$ ./memtool -32 40000000 10
Reading 0x10 count starting at address 0x40000000

0x40000000:  DEADBEAF 00000000 00000000 00000000
0x40000010:  00000000 00000000 00000000 00000000
0x40000020:  00000000 00000000 00000000 00000000
0x40000030:  00000000 00000000 00000000 00000000


3 people found this helpful

Outcomes