AnsweredAssumed Answered

T1040 L2 switch frame fragmentation problem

Question asked by Szymon Kukliński on May 25, 2017
Latest reply on Jun 2, 2017 by Yiping Wang

Hi

 

During our tests we found out that the integrated Ethernet switch internal port has problems with handling a specific traffic type. We established that the switch drops frames arriving at chip_port 9 and marks them as fragments (rmon.rx_etherStatsFragments in vtss_port_counters_t struct according to Microsemi API). Interestingly this problem does not occur on other ports (we tested the exact same traffic type between on chip_ports 0 and 1).

 

It is very easy to recreate this problem (as seen between chip_ports 0 and 8):

# ping6 2001::2 -c1 -s20000

PING 2001::2(2001::2) 20000 data bytes

--- 2001::2 ping statistics ---

1 packets transmitted, 0 received, 100% packet loss, time 0ms

 

expected behaviour (as seen between chip_ports 0 and 1):

# ping6 2001::2 -c1 -s20000

PING 2001::2(2001::2) 20000 data bytes

20008 bytes from 2001::2: icmp_seq=1 ttl=64 time=0.932 ms

--- 2001::2 ping statistics ---

1 packets transmitted, 1 received, 0% packet loss, time 0ms

rtt min/avg/max/mdev = 0.932/0.932/0.932/0.000 ms

 

I measured that the maximum data bytes size is 15920 for ping6 and 16272 for ping. Every ping larger than that gets dropped and caused rmon.rx_etherStatsFragments to increment. The intrrface from which the frame originates is configured as 802.1ad:

ip -d link show mgmt
9: mgmt@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 00:80:ea:0f:24:90 brd ff:ff:ff:ff:ff:ff promiscuity 0
vlan protocol 802.1ad id 100 <REORDER_HDR> numtxqueues 1 numrxqueues 1

 

The internal switch is configured by our own driver based on NXP l2switch program and our previous experience with Vitesse/Microsemi switches. The most crucial port settings are:

1. vtss_port_map_t 
{
/* chip_port, chip_no, miim_controller, miim_addr, miim_chip_no */
{0, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //lower slot
{1, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //upper slot
{2, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{3, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{8, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //CPU
{9, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}, //CPU
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0},
{CHIP_PORT_UNUSED, 0, VTSS_MIIM_CONTROLLER_NONE, 0, 0}
};

 

2. vtss_port_conf_t:

/* if_type, sd_enable, sd_active_high, sd_internal, frame_gaps, power_down, speed,
* fdx, flow_control, max_frame_length
* max_tags, exc_col_cont, xaui_rx_lane_flip, xaui_tx_lane_flip, loop, serdes */

chip_port 0:
{VTSS_PORT_INTERFACE_SGMII, 0, 0, 0, {0, 0, 0}, 0, VTSS_SPEED_1G,
1, {1, 1, {0}, {0}}, VTSS_MAX_FRAME_LENGTH_MAX,
VTSS_PORT_MAX_TAGS_TWO, 0, 0, 0, VTSS_PORT_LOOP_DISABLE, {1}},

(chip_port 1, chip_port 2 and chip_port 3 are configured as chip_port 0)

chip_port 8:

{VTSS_PORT_INTERFACE_VAUI, 0, 0, 0, {0, 0, 0}, 0, VTSS_SPEED_2500M,
1, {1, 1, {0}, {0}}, VTSS_MAX_FRAME_LENGTH_MAX,
VTSS_PORT_MAX_TAGS_TWO, 0, 0, 0, VTSS_PORT_LOOP_DISABLE, {1}},

chip_port 9:

{VTSS_PORT_INTERFACE_VAUI, 0, 0, 0, {0, 0, 0}, 0, VTSS_SPEED_2500M,
1, {1, 1, {0}, {0}}, VTSS_MAX_FRAME_LENGTH_MAX,
VTSS_PORT_MAX_TAGS_TWO, 0, 0, 0, VTSS_PORT_LOOP_DISABLE, {1}},

 

Kind regards

Szymon Kukliński

Outcomes