T1040 L2 switch frame fragmentation problem

[elszymono]

Hi

During our tests we found out that the integrated Ethernet switch internal port has problems with handling a specific traffic type. We established that the switch drops frames arriving at chip_port 9 and marks them as fragments (rmon.rx_etherStatsFragments in vtss_port_counters_t struct according to Microsemi API). Interestingly this problem does not occur on other ports (we tested the exact same traffic type between on chip_ports 0 and 1).

It is very easy to recreate this problem (as seen between chip_ports 0 and 8):

# ping6 2001::2 -c1 -s20000

PING 2001::2(2001::2) 20000 data bytes

--- 2001::2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms

expected behaviour (as seen between chip_ports 0 and 1):

# ping6 2001::2 -c1 -s20000

PING 2001::2(2001::2) 20000 data bytes

20008 bytes from 2001::2: icmp_seq=1 ttl=64 time=0.932 ms

--- 2001::2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.932/0.932/0.932/0.000 ms

I measured that the maximum data bytes size is 15920 for ping6 and 16272 for ping. Every ping larger than that gets dropped and caused rmon.rx_etherStatsFragments to increment. The intrrface from which the frame originates is configured as 802.1ad:

ip -d link show mgmt
9: mgmt@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 00:80:ea:0f:24:90 brd ff:ff:ff:ff:ff:ff promiscuity 0
vlan protocol 802.1ad id 100 <REORDER_HDR> numtxqueues 1 numrxqueues 1

The internal switch is configured by our own driver based on NXP l2switch program and our previous experience with Vitesse/Microsemi switches. The most crucial port settings are:

1. vtss_port_map_t 
{
/* chip_port, chip_no, miim_controller, miim_addr, miim_chip_no */
{0, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //lower slot
{1, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //upper slot
{2, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{3, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{8, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //CPU
{9, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //CPU
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}
};

2. vtss_port_conf_t:

/* if_type, sd_enable, sd_active_high, sd_internal, frame_gaps, power_down, speed,
* fdx, flow_control, max_frame_length
* max_tags, exc_col_cont, xaui_rx_lane_flip, xaui_tx_lane_flip, loop, serdes */

chip_port 0:
{VTSS_PORT_INTERFACE_SGMII, 0, 0, 0, {0, 0, 0}, 0, VTSS_SPEED_1G,
1, {1, 1, {0}, {0}}, VTSS_MAX_FRAME_LENGTH_MAX,
VTSS_PORT_MAX_TAGS_TWO, 0, 0, 0, VTSS_PORT_LOOP_DISABLE, {1}},

(chip_port 1, chip_port 2 and chip_port 3 are configured as chip_port 0)

chip_port 8:

{VTSS_PORT_INTERFACE_VAUI, 0, 0, 0, {0, 0, 0}, 0, VTSS_SPEED_2500M,
1, {1, 1, {0}, {0}}, VTSS_MAX_FRAME_LENGTH_MAX,
VTSS_PORT_MAX_TAGS_TWO, 0, 0, 0, VTSS_PORT_LOOP_DISABLE, {1}},

chip_port 9:

{VTSS_PORT_INTERFACE_VAUI, 0, 0, 0, {0, 0, 0}, 0, VTSS_SPEED_2500M,
1, {1, 1, {0}, {0}}, VTSS_MAX_FRAME_LENGTH_MAX,
VTSS_PORT_MAX_TAGS_TWO, 0, 0, 0, VTSS_PORT_LOOP_DISABLE, {1}},

Kind regards

Szymon Kukliński

[yipingwang]

Hello Szymon Kukliński,

Please try whether increasing the value VTSS_MAX_FRAME_LENGTH_MAX according to your requirement would be helpful. In the default config, it is defined as "10240" in include/vtss_api/vtss_port_api.h of l2switch source code.

Have a great day,
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------