LS1046ARDB - maximum throughput for two10G ethernet ports

[masonyen]

LS1046ARDB, when using iperf to stress two 10G ports at the same time, only gets 7.x Gbps for each 10G port. What's the maximum throughput for stressing two 10G ports at the same time? Is it possible to reach 20Gbps with two 10G ports?

fm1-mac9

fm1-mac10

Cheers,

Mason

[masonyen]

Hi Yiping,

For LS1046A-RDB, if stressing fm1-mac10(10G SFP+) or fm1-mac9(10G Copper) individually, performance is around 9.x Gbits for one 10G port. But if stressing both 10G SPF+ and 10G Copper at the same time,  performance is around 7.x Gbits per 10G ports(Copper or SPF+). Theoretically, it should be around 19.X Gbits(10G+10Gbits).  What's the maximum performance if stressing both 10G Copper and 10G SPF+ on LS1046A-RDB at the same time?

Cheers,

Mason

[yipingwang]

Hello Mason,

I checked the test data from the testing team.

In  LSDK 19.09 4 core 20G Netperf testing on LS1046ARDB, with Linux Kernel 4.19 the max test data is about 15822.7 Mbps.

Thanks,

Yiping

[masonyen]

Hi Yiping,

It is 15822.7 Mbps instead of 20Gbps. What's the bottleneck of 2*10G netperf test?

Is it possible to reach 20 Gbps by using any software settings, such as IP forwarding?

Cheers,

Mason

[yipingwang]

Hello Mason,

For 2 cores 20 G on LS1046ARDB, the max netperf through put is 9.105Gbps, so the bottleneck is the number of processors.

4 cores 20G IP forward performance is as the following.

FrameSize(Bytes)            Throughput (Mbps)

64                                      2029.71

128                                    3551.62

256                                    6614.96                       

512                                    9756.35               

390                                    12605.06                        

1024                                  19980.49                     

1280                                  19980.49          

1518                                  19980.49

Thanks,

Yiping

[Bruno_WT]

Hi, YiPing,

Do you have the test result on LSDK2108?

I had disabled these options "CONFIG_NETFILTER" and "CONFIG_CPU_FREQ" at LS1046ARDB.

But I only gets 16.4Gbps. Do you have any suggestion?

[masonyen]

Hi Yiping,

How do I enable 4 cores 20G IP forward? What's your OS version and settings?

Below is my result with IP forward enabled. The throughput is not as good as yours.

Avg Frame Size	Throughput (%)
64	6.052
128	10.694
256	19.828
512	38.109
1024	75.262
1280	92.074
1518	99.87

[yipingwang]

Hello Mason,

Please use images built from the latest LSDK 1909.

Please build Linux Kernel image with options CONFIG_NETFILTER and CONFIG_CPU_FREQ disabled.

Please use Ubuntu rootfs filesystem.

Thanks,

Yiping

[masonyen]

Hi Yiping,

I tried to remove below CONFIG_CPU_FREQ in arch/arm64/configs/defconfig from flexbuild_lsdk1909_update_311219. But all fman-port failed during POST. Please see below text. Any idea to fix?

CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
CONFIG_CPU_FREQ_GOV_SCHEDUTIL=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=y
CONFIG_CPUFREQ_DT=y
CONFIG_ACPI_CPPC_CPUFREQ=m
CONFIG_ARM_ARMADA_37XX_CPUFREQ=y
CONFIG_ARM_BIG_LITTLE_CPUFREQ=y
CONFIG_ARM_SCPI_CPUFREQ=y
CONFIG_ARM_TEGRA186_CPUFREQ=y
CONFIG_ACPI_CPPC_CPUFREQ=m
CONFIG_QORIQ_CPUFREQ=y

[    2.539738] fsl-fman-port: probe of 1a82000.port failed with error -5
[    2.546206] fsl-fman-port: probe of 1a83000.port failed with error -5
[    2.552676] fsl-fman-port: probe of 1a84000.port failed with error -5
[    2.559143] fsl-fman-port: probe of 1a85000.port failed with error -5
[    2.565612] fsl-fman-port: probe of 1a86000.port failed with error -5
[    2.572080] fsl-fman-port: probe of 1a87000.port failed with error -5
[    2.578560] fsl-fman-port: probe of 1a88000.port failed with error -5
[    2.585029] fsl-fman-port: probe of 1aa8000.port failed with error -5
[    2.591505] fsl-fman-port: probe of 1a89000.port failed with error -5
[    2.597975] fsl-fman-port: probe of 1aa9000.port failed with error -5
[    2.604445] fsl-fman-port: probe of 1a8a000.port failed with error -5
[    2.610915] fsl-fman-port: probe of 1aaa000.port failed with error -5
[    2.617387] fsl-fman-port: probe of 1a8b000.port failed with error -5
[    2.623856] fsl-fman-port: probe of 1aab000.port failed with error -5
[    2.630328] fsl-fman-port: probe of 1a8c000.port failed with error -5
[    2.636800] fsl-fman-port: probe of 1aac000.port failed with error -5
[    2.643272] fsl-fman-port: probe of 1a8d000.port failed with error -5
[    2.649743] fsl-fman-port: probe of 1aad000.port failed with error -5
[    2.656214] fsl-fman-port: probe of 1a90000.port failed with error -5
[    2.662684] fsl-fman-port: probe of 1ab0000.port failed with error -5
[    2.669155] fsl-fman-port: probe of 1a91000.port failed with error -5
[    2.675623] fsl-fman-port: probe of 1ab1000.port failed with error -5
[    2.682092] Freescale FM Ports module
[    2.685753] fsl_mac: fsl_mac: FSL FMan MAC API based driver
[    2.691433] fsl_mac 1ae4000.ethernet: fm_bind(fman@1a00000) failed
[    2.697649] fsl_mac 1ae6000.ethernet: fm_bind(fman@1a00000) failed
[    2.703860] fsl_mac 1ae8000.ethernet: fm_bind(fman@1a00000) failed
[    2.710074] fsl_mac 1aea000.ethernet: fm_bind(fman@1a00000) failed
[    2.716284] fsl_mac 1af0000.ethernet: fm_bind(fman@1a00000) failed
[    2.722498] fsl_mac 1af2000.ethernet: fm_bind(fman@1a00000) failed
[    2.728701] fsl_dpa: FSL DPAA Ethernet driver
[    2.733179] fsl_dpa soc:fsl,dpaa:ethernet@2: dev_get_drvdata(1ae4000.ethernet) failed
[    2.741019] fsl_dpa: probe of soc:fsl,dpaa:ethernet@2 failed with error -22
[    2.748021] fsl_dpa soc:fsl,dpaa:ethernet@3: dev_get_drvdata(1ae6000.ethernet) failed
[    2.755860] fsl_dpa: probe of soc:fsl,dpaa:ethernet@3 failed with error -22
[    2.762863] fsl_dpa soc:fsl,dpaa:ethernet@4: dev_get_drvdata(1ae8000.ethernet) failed
[    2.770702] fsl_dpa: probe of soc:fsl,dpaa:ethernet@4 failed with error -22
[    2.777704] fsl_dpa soc:fsl,dpaa:ethernet@5: dev_get_drvdata(1aea000.ethernet) failed
[    2.785543] fsl_dpa: probe of soc:fsl,dpaa:ethernet@5 failed with error -22
[    2.792547] fsl_dpa soc:fsl,dpaa:ethernet@8: dev_get_drvdata(1af0000.ethernet) failed
[    2.800386] fsl_dpa: probe of soc:fsl,dpaa:ethernet@8 failed with error -22
[    2.807390] fsl_dpa soc:fsl,dpaa:ethernet@9: dev_get_drvdata(1af2000.ethernet) failed
[    2.815228] fsl_dpa: probe of soc:fsl,dpaa:ethernet@9 failed with error -22

Cheers,

Mason

[yipingwang]

Please go to folder build/linux/linux/arm64/LS/output/LSDK-19.09-update-311219-V4.19, run "make ARCH=arm64 menuconfig", then disable CONFIG_NETFILTER and CONFIG_CPU_FREQ.

Rebuild Linux with the following command.

$ flex-builder -c linux -a arm64

Please get  built Kernel image in build/linux/kernel/arm64/LS/Image

Thanks,

Yiping

[masonyen]

Hi Yiping,

I did use menuconfig and build with -c linux. I also tried to rebuild ubuntu rfs. Still meet the same error.

Cheers,

Mason

[yipingwang]

Please refer to the attached Linux Kernel configuration file, which I have verified on LS1046ARDB.

[masonyen]

Hi Yiping,

After using your kernel config file, it is able to disable CONFIG_NETFILTER and CONFIG_CPU_FREQ. And the 2 * 10G performance can reach 19.878Gbps. Thanks.

Cheers,

Mason

[yipingwang]

Would you please provide your Linux Kernel configuration file?

[yipingwang]

Hello Mason Yen,

I used two LS1046ARDB boards connecting through the copper port(fm1-mac9), I got 9.38 Gbits/sec performance on the client side.

Please refer to the following.

1.  Please run the following iperf commands on the server and client sides.

Sever: iperf  -s

Client:

iperf -c 10.10.10.2 -P 10 -t 30

 2. When doing iperf performance test, please compile Linux Kernel with CONFIG_NETFILTER and CONFIG_CPU_FREQ disabled.

3. Please use Ubuntu rootfs filesystem to boot the LS1046ARDB.

Thanks,

Yiping