I am seeing a throughput issue with transmit on a SGMII interface if the Topaz hypervisor is used. This happens on T2080 processor and in hypervisor configuration a single guest partition is assigned all CPU cores and all the interfaces.
Following is the setup:
- FSL SDK 2.0 (Uboot, kernel, hypervisor)
- One of the SGMII interfaces of T2080 is connected to another processor's SGMII interface. So, there is a direct connection between two processors.
- Same boot images (uboot and kernels) are used in SMP(Symmetric Multiprocessing) mode as well with Hypervisor.
- iperf version 2.0.9 is used to measure TCP throughput.
Before running the test, detail NIC statistics were gathered in SMP mode as well as hypervisor mode and attached here "SMP_before_transmit_receive_eth6.txt", HV_before_transmit_receive_eth6.txt.
In SMP mode (No hypervisor)
In SMP mode, the iperf client with four threads(-P 4) was ran on processor card with T2080 and iperf server was ran on another processor card connected with T2080 over SGMII. In this test, the throughput was in range of 850 Mbps. I captured the detailed statistic after doing the test and attached here (SMP_after_transmit_eth6.txt).
With Hypervisor mode (one guest and all cores are allocated to one core)
When booting one guest on hypervisor, the iperf client with four threads was ran on processor card with T2080 and iperf server was ran on another processor card connected with T2080 over SGMII. In this test, the throughput was in range of 540 Mbps. I captured the detailed statistic after doing the test and attached here (HV_after_transmit_eth6.txt).
Receive throughput is not affected (similar throughput with and without Hypervisor)
I see similar throughput for receive in both SMP and Hypervisor modes. In these tests, iperf server was ran on processor card with T2080 and iperf client with four threads was ran on the processor connected with T2080 over SGMII. I also attached NIC statistics collected after running the test in SMP and hypervisor modes(SMP_after_receive_eth6.txt, HV_after_receive_eth6.txt).
I would like to understand that why the transmit throughput is affected if used with Topaz hypervisor.