topic Re: Ethernet performance on FRWY-LS1012A in Layerscape

Ethernet performance on FRWY-LS1012A

tomasz_zyjewski — Wed, 09 Sep 2020 07:50:37 GMT

Hi
I am using QorIQ FRWY-LS1012A reference design board with LSDK v20.04. In order to run sdk I download `flex-installer`

$ wget https://www.nxp.com/lgfiles/sdk/lsdk2004/flex-installer && chmod +x flex-installer && sudo mv flex-installer /usr/bin

and flash sd card

$ flex-installer -i auto -m ls1012afrwy -d /dev/sdc

I stopped pfe in U-Boot (to use interfaces in Linux) and run bootcmd

=> pfe stop Stopping PFE... => run bootcmd switch to partitions #0, OK mmc0 is current device Scanning mmc 0:1... Scanning mmc 0:2... Found U-Boot script /ls1012afrwy_boot.scr 964 bytes read in 321 ms (2.9 KiB/s) ## Executing script at 80000000 36745728 bytes read in 2183 ms (16.1 MiB/s) 15316 bytes read in 106 ms (140.6 KiB/s) ## Flattened Device Tree blob at 90000000 Booting using the fdt blob at 0x90000000 Using Device Tree in place at 0000000090000000, end 0000000090006bd3 WARNING: fdt_fixup_stdout: could not read serial0 alias: FDT_ERR_NOTFOUND PCIe0: pcie@3400000 Root Complex: no link Starting kernel ...

To use eth1 interface I connect ethernet cable and run `dhclient` on that interface

root@localhost:~# dhclient -i eth0

According to the documentation frwy-ls1012a should be able to receive ~1Gbps data. I run some tests using `iperf3` and I get only ~600mbps.

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 72.1 MBytes 602 Mbits/sec [ 5] 1.00-2.00 sec 73.9 MBytes 619 Mbits/sec [ 5] 2.00-3.00 sec 73.8 MBytes 620 Mbits/sec [ 5] 3.00-4.00 sec 73.4 MBytes 616 Mbits/sec [ 5] 4.00-5.00 sec 73.6 MBytes 618 Mbits/sec [ 5] 5.00-6.00 sec 73.2 MBytes 615 Mbits/sec [ 5] 6.00-7.00 sec 73.2 MBytes 615 Mbits/sec [ 5] 7.00-8.00 sec 73.2 MBytes 614 Mbits/sec [ 5] 8.00-9.00 sec 73.0 MBytes 613 Mbits/sec [ 5] 9.00-10.00 sec 72.9 MBytes 611 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.00 sec 732 MBytes 614 Mbits/sec receiver -----------------------------------------------------------

Is there any way to improve this performance?

Best regards,
Tom

Re: Ethernet performance on FRWY-LS1012A

andrei_skok — Wed, 16 Sep 2020 03:50:26 GMT

For LS1012A, in order to get good performance, you need to enable PFE.
Without the PFE acceleration, IP forwarding performance won't be great.

Re: Ethernet performance on FRWY-LS1012A

tomasz_zyjewski — Wed, 16 Sep 2020 05:08:23 GMT

Hi @andrei_skok , thanks for reply.

Could you please tell me how I can enable PFE in Linux or how I can check if it is enabled?
From `dmesg` I can tell that PFE firmware is loaded

# dmesg | grep pfe [ 6.682160] pfe: module is from the staging directory, the quality is unknown, you have been warned. [ 6.807076] pfe_hw_init [ 6.956846] pfe_hif_lib_init [ 6.964048] pfe_hif_init [ 6.966593] pfe_hif_alloc_descr [ 6.976699] pfe_hif_init_buffers [ 6.984766] pfe_firmware_init [ 6.997622] pfe_load_elf [ 7.062211] PFE binary version: pfe_ls1012a_slowpath_07-dirty [ 7.076084] pfe_load_elf [ 7.079608] pfe_ctrl_init [ 7.090882] pfe_ctrl_init finished [ 7.097899] pfe_eth_init [ 7.103782] pfe 4000000.pfe: pfe_eth_mdio_init: mdc_div: 64, phy_mask: ffffffff [ 7.160801] pfe 4000000.pfe: pfe_eth_mdio_init: mdc_div: 64, phy_mask: ffffffff [ 7.188691] pfe_phy_init interface 4 [ 7.196934] pfe 4000000.pfe eth0: pfe_eth_init_one: created interface, baseaddr: (____ptrval____) [ 7.218902] pfe_phy_init interface 4 [ 7.227784] pfe 4000000.pfe eth1: pfe_eth_init_one: created interface, baseaddr: (____ptrval____) [ 7.245513] pfe_debugfs_init [ 14.831098] pfe_eth_gemac_reglen() [ 14.839588] pfe_eth_gemac_reglen() [ 31.209172] pfe 4000000.pfe eth1: pfe_eth_open [ 31.209368] pfe_hif_client_register [ 31.209376] pfe 4000000.pfe eth1: pfe_gemac_init [ 31.210004] pfe 4000000.pfe eth1: Link is Up - 1Gbps/Full - flow control off

Or maybe I should not stop PFE in U-Boot? But if I not do that, I am not able to get any interface to work.
Any hints how to enable PFE acceleration will be helpful.

Re: Ethernet performance on FRWY-LS1012A

andrei_skok — Mon, 21 Sep 2020 11:46:20 GMT

Please look at the https://community.nxp.com/t5/Layerscape/LS1012A-How-to-enable-Packet-Forwarding-Engine-PFE/td-p/1153413

Re: Ethernet performance on FRWY-LS1012A

tomasz_zyjewski — Mon, 21 Sep 2020 11:55:25 GMT

Thanks for that link, will certainly check that and let you know if it helps me.