Ethernet performance on FRWY-LS1012A

tomasz_zyjewski · ‎09-09-2020

Hi
I am using QorIQ FRWY-LS1012A reference design board with LSDK v20.04. In order to run sdk I download `flex-installer`

$ wget https://www.nxp.com/lgfiles/sdk/lsdk2004/flex-installer && chmod +x flex-installer && sudo mv flex-installer /usr/bin

and flash sd card

$ flex-installer -i auto -m ls1012afrwy -d /dev/sdc

I stopped pfe in U-Boot (to use interfaces in Linux) and run bootcmd

=> pfe stop
Stopping PFE...
=> run bootcmd
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Scanning mmc 0:2...
Found U-Boot script /ls1012afrwy_boot.scr
964 bytes read in 321 ms (2.9 KiB/s)
## Executing script at 80000000
36745728 bytes read in 2183 ms (16.1 MiB/s)
15316 bytes read in 106 ms (140.6 KiB/s)
## Flattened Device Tree blob at 90000000
Booting using the fdt blob at 0x90000000
Using Device Tree in place at 0000000090000000, end 0000000090006bd3
WARNING: fdt_fixup_stdout: could not read serial0 alias: FDT_ERR_NOTFOUND
PCIe0: pcie@3400000 Root Complex: no link

Starting kernel ...

To use eth1 interface I connect ethernet cable and run `dhclient` on that interface

root@localhost:~# dhclient -i eth0

According to the documentation frwy-ls1012a should be able to receive ~1Gbps data. I run some tests using `iperf3` and I get only ~600mbps.

[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 72.1 MBytes 602 Mbits/sec
[ 5] 1.00-2.00 sec 73.9 MBytes 619 Mbits/sec
[ 5] 2.00-3.00 sec 73.8 MBytes 620 Mbits/sec
[ 5] 3.00-4.00 sec 73.4 MBytes 616 Mbits/sec
[ 5] 4.00-5.00 sec 73.6 MBytes 618 Mbits/sec
[ 5] 5.00-6.00 sec 73.2 MBytes 615 Mbits/sec
[ 5] 6.00-7.00 sec 73.2 MBytes 615 Mbits/sec
[ 5] 7.00-8.00 sec 73.2 MBytes 614 Mbits/sec
[ 5] 8.00-9.00 sec 73.0 MBytes 613 Mbits/sec
[ 5] 9.00-10.00 sec 72.9 MBytes 611 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.00 sec 732 MBytes 614 Mbits/sec receiver
-----------------------------------------------------------

Is there any way to improve this performance?

Best regards,
Tom

andrei_skok · ‎09-15-2020

For LS1012A, in order to get good performance, you need to enable PFE.
Without the PFE acceleration, IP forwarding performance won't be great.

tomasz_zyjewski · ‎09-15-2020

Hi @andrei_skok , thanks for reply.

Could you please tell me how I can enable PFE in Linux or how I can check if it is enabled?
From `dmesg` I can tell that PFE firmware is loaded

# dmesg | grep pfe
[    6.682160] pfe: module is from the staging directory, the quality is unknown, you have been warned.
[    6.807076] pfe_hw_init
[    6.956846] pfe_hif_lib_init
[    6.964048] pfe_hif_init
[    6.966593] pfe_hif_alloc_descr
[    6.976699] pfe_hif_init_buffers
[    6.984766] pfe_firmware_init
[    6.997622] pfe_load_elf
[    7.062211] PFE binary version: pfe_ls1012a_slowpath_07-dirty
[    7.076084] pfe_load_elf
[    7.079608] pfe_ctrl_init
[    7.090882] pfe_ctrl_init finished
[    7.097899] pfe_eth_init
[    7.103782] pfe 4000000.pfe: pfe_eth_mdio_init: mdc_div: 64, phy_mask: ffffffff
[    7.160801] pfe 4000000.pfe: pfe_eth_mdio_init: mdc_div: 64, phy_mask: ffffffff
[    7.188691] pfe_phy_init interface 4
[    7.196934] pfe 4000000.pfe eth0: pfe_eth_init_one: created interface, baseaddr: (____ptrval____)
[    7.218902] pfe_phy_init interface 4
[    7.227784] pfe 4000000.pfe eth1: pfe_eth_init_one: created interface, baseaddr: (____ptrval____)
[    7.245513] pfe_debugfs_init
[   14.831098] pfe_eth_gemac_reglen()
[   14.839588] pfe_eth_gemac_reglen()
[   31.209172] pfe 4000000.pfe eth1: pfe_eth_open
[   31.209368] pfe_hif_client_register
[   31.209376] pfe 4000000.pfe eth1: pfe_gemac_init
[   31.210004] pfe 4000000.pfe eth1: Link is Up - 1Gbps/Full - flow control off

Or maybe I should not stop PFE in U-Boot? But if I not do that, I am not able to get any interface to work.
Any hints how to enable PFE acceleration will be helpful.

andrei_skok · ‎09-21-2020

Please look at the https://community.nxp.com/t5/Layerscape/LS1012A-How-to-enable-Packet-Forwarding-Engine-PFE/td-p/1153...

tomasz_zyjewski · ‎09-21-2020

Thanks for that link, will certainly check that and let you know if it helps me.