topic Re: LS1046ARDB - maximum throughput for two10G ethernet ports in Layerscape

LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Thu, 02 Apr 2020 18:32:38 GMT

LS1046ARDB, when using iperf to stress two 10G ports at the same time, only gets 7.x Gbps for each 10G port. What's the maximum throughput for stressing two 10G ports at the same time? Is it possible to reach 20Gbps with two 10G ports?

fm1-mac9

fm1-mac10

Cheers,

Mason

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Fri, 03 Apr 2020 04:26:19 GMT

Hello Mason Yen,

I used two LS1046ARDB boards connecting through the copper port(fm1-mac9), I got 9.38 Gbits/sec performance on the client side.

Please refer to the following.

1. Please run the following iperf commands on the server and client sides.

Sever: iperf -s

Client:

iperf -c 10.10.10.2 -P 10 -t 30

2. When doing iperf performance test, please compile Linux Kernel with CONFIG_NETFILTER and CONFIG_CPU_FREQ disabled.

3. Please use Ubuntu rootfs filesystem to boot the LS1046ARDB.

Thanks,

Yiping

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Mon, 06 Apr 2020 07:14:50 GMT

Hi Yiping,

For LS1046A-RDB, if stressing fm1-mac10(10G SFP+) or fm1-mac9(10G Copper) individually, performance is around 9.x Gbits for one 10G port. But if stressing both 10G SPF+ and 10G Copper at the same time, performance is around 7.x Gbits per 10G ports(Copper or SPF+). Theoretically, it should be around 19.X Gbits(10G+10Gbits). What's the maximum performance if stressing both 10G Copper and 10G SPF+ on LS1046A-RDB at the same time?

Cheers,

Mason

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Tue, 07 Apr 2020 07:43:26 GMT

Hello Mason,

I checked the test data from the testing team.

In LSDK 19.09 4 core 20G Netperf testing on LS1046ARDB, with Linux Kernel 4.19 the max test data is about 15822.7 Mbps.

Thanks,

Yiping

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Tue, 07 Apr 2020 08:32:23 GMT

Hi Yiping,

It is 15822.7 Mbps instead of 20Gbps. What's the bottleneck of 2*10G netperf test?

Is it possible to reach 20 Gbps by using any software settings, such as IP forwarding?

Cheers,

Mason

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Wed, 08 Apr 2020 04:34:10 GMT

Hello Mason,

For 2 cores 20 G on LS1046ARDB, the max netperf through put is 9.105Gbps, so the bottleneck is the number of processors.

4 cores 20G IP forward performance is as the following.

FrameSize(Bytes) Throughput (Mbps)

64 2029.71

128 3551.62

256 6614.96

512 9756.35

390 12605.06

1024 19980.49

1280 19980.49

1518 19980.49

Thanks,

Yiping

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Sun, 12 Apr 2020 15:00:52 GMT

Hi Yiping,

How do I enable 4 cores 20G IP forward? What's your OS version and settings?

Below is my result with IP forward enabled. The throughput is not as good as yours.

Avg Frame Size	Throughput (%)
64	6.052
128	10.694
256	19.828
512	38.109
1024	75.262
1280	92.074
1518	99.87

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Mon, 13 Apr 2020 05:18:19 GMT

Hello Mason,

Please use images built from the latest LSDK 1909.

Please build Linux Kernel image with options CONFIG_NETFILTER and CONFIG_CPU_FREQ disabled.

Please use Ubuntu rootfs filesystem.

Thanks,

Yiping

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Mon, 13 Apr 2020 07:26:32 GMT

Hi Yiping,

I tried to remove below CONFIG_CPU_FREQ in arch/arm64/configs/defconfig from flexbuild_lsdk1909_update_311219. But all fman-port failed during POST. Please see below text. Any idea to fix?

CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
CONFIG_CPUFREQ_DT=y
CONFIG_ACPI_CPPC_CPUFREQ=m
CONFIG_ARM_ARMADA_37XX_CPUFREQ=y
CONFIG_ARM_BIG_LITTLE_CPUFREQ=y
CONFIG_ARM_SCPI_CPUFREQ=y
CONFIG_ARM_TEGRA186_CPUFREQ=y
CONFIG_ACPI_CPPC_CPUFREQ=m
CONFIG_QORIQ_CPUFREQ=y

[    2.539738] fsl-fman-port: probe of 1a82000.port failed with error -5
[    2.546206] fsl-fman-port: probe of 1a83000.port failed with error -5
[    2.552676] fsl-fman-port: probe of 1a84000.port failed with error -5
[    2.559143] fsl-fman-port: probe of 1a85000.port failed with error -5
[    2.565612] fsl-fman-port: probe of 1a86000.port failed with error -5
[    2.572080] fsl-fman-port: probe of 1a87000.port failed with error -5
[    2.578560] fsl-fman-port: probe of 1a88000.port failed with error -5
[    2.585029] fsl-fman-port: probe of 1aa8000.port failed with error -5
[    2.591505] fsl-fman-port: probe of 1a89000.port failed with error -5
[    2.597975] fsl-fman-port: probe of 1aa9000.port failed with error -5
[    2.604445] fsl-fman-port: probe of 1a8a000.port failed with error -5
[    2.610915] fsl-fman-port: probe of 1aaa000.port failed with error -5
[    2.617387] fsl-fman-port: probe of 1a8b000.port failed with error -5
[    2.623856] fsl-fman-port: probe of 1aab000.port failed with error -5
[    2.630328] fsl-fman-port: probe of 1a8c000.port failed with error -5
[    2.636800] fsl-fman-port: probe of 1aac000.port failed with error -5
[    2.643272] fsl-fman-port: probe of 1a8d000.port failed with error -5
[    2.649743] fsl-fman-port: probe of 1aad000.port failed with error -5
[    2.656214] fsl-fman-port: probe of 1a90000.port failed with error -5
[    2.662684] fsl-fman-port: probe of 1ab0000.port failed with error -5
[    2.669155] fsl-fman-port: probe of 1a91000.port failed with error -5
[    2.675623] fsl-fman-port: probe of 1ab1000.port failed with error -5
[    2.682092] Freescale FM Ports module
[    2.685753] fsl_mac: fsl_mac: FSL FMan MAC API based driver
[    2.691433] fsl_mac 1ae4000.ethernet: fm_bind(fman@1a00000) failed
[    2.697649] fsl_mac 1ae6000.ethernet: fm_bind(fman@1a00000) failed
[    2.703860] fsl_mac 1ae8000.ethernet: fm_bind(fman@1a00000) failed
[    2.710074] fsl_mac 1aea000.ethernet: fm_bind(fman@1a00000) failed
[    2.716284] fsl_mac 1af0000.ethernet: fm_bind(fman@1a00000) failed
[    2.722498] fsl_mac 1af2000.ethernet: fm_bind(fman@1a00000) failed
[    2.728701] fsl_dpa: FSL DPAA Ethernet driver
[    2.733179] fsl_dpa soc:fsl,dpaa:ethernet@2: dev_get_drvdata(1ae4000.ethernet) failed
[    2.741019] fsl_dpa: probe of soc:fsl,dpaa:ethernet@2 failed with error -22
[    2.748021] fsl_dpa soc:fsl,dpaa:ethernet@3: dev_get_drvdata(1ae6000.ethernet) failed
[    2.755860] fsl_dpa: probe of soc:fsl,dpaa:ethernet@3 failed with error -22
[    2.762863] fsl_dpa soc:fsl,dpaa:ethernet@4: dev_get_drvdata(1ae8000.ethernet) failed
[    2.770702] fsl_dpa: probe of soc:fsl,dpaa:ethernet@4 failed with error -22
[    2.777704] fsl_dpa soc:fsl,dpaa:ethernet@5: dev_get_drvdata(1aea000.ethernet) failed
[    2.785543] fsl_dpa: probe of soc:fsl,dpaa:ethernet@5 failed with error -22
[    2.792547] fsl_dpa soc:fsl,dpaa:ethernet@8: dev_get_drvdata(1af0000.ethernet) failed
[    2.800386] fsl_dpa: probe of soc:fsl,dpaa:ethernet@8 failed with error -22
[    2.807390] fsl_dpa soc:fsl,dpaa:ethernet@9: dev_get_drvdata(1af2000.ethernet) failed
[    2.815228] fsl_dpa: probe of soc:fsl,dpaa:ethernet@9 failed with error -22

Cheers,

Mason

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Mon, 13 Apr 2020 08:30:07 GMT

Please go to folder build/linux/linux/arm64/LS/output/LSDK-19.09-update-311219-V4.19, run "make ARCH=arm64 menuconfig", then disable CONFIG_NETFILTER and CONFIG_CPU_FREQ.

Rebuild Linux with the following command.

$ flex-builder -c linux -a arm64

Please get built Kernel image in build/linux/kernel/arm64/LS/Image

Thanks,

Yiping

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Mon, 13 Apr 2020 09:04:13 GMT

Hi Yiping,

I did use menuconfig and build with -c linux. I also tried to rebuild ubuntu rfs. Still meet the same error.

Cheers,

Mason

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Mon, 13 Apr 2020 09:24:05 GMT

Would you please provide your Linux Kernel configuration file?

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

yipingwang — Tue, 14 Apr 2020 07:17:22 GMT

Please refer to the attached Linux Kernel configuration file, which I have verified on LS1046ARDB.

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

masonyen — Wed, 22 Apr 2020 02:52:57 GMT

Hi Yiping,

After using your kernel config file, it is able to disable CONFIG_NETFILTER and CONFIG_CPU_FREQ. And the 2 * 10G performance can reach 19.878Gbps. Thanks.

Cheers,

Mason

Re: LS1046ARDB - maximum throughput for two10G ethernet ports

Bruno_WT — Fri, 16 Jun 2023 07:13:46 GMT

Hi, YiPing,

Do you have the test result on LSDK2108?

I had disabled these options "CONFIG_NETFILTER" and "CONFIG_CPU_FREQ" at LS1046ARDB.

But I only gets 16.4Gbps. Do you have any suggestion?