How to configure switch LS1028ARDB to use the Frame Replication and Elimination for Reliability ?

[asanson]

Good afternoon, 

I would like to create a redundant network using the TSN 802.1 CB standard (Frame Replication and Elimination for Reliabiliy) and the switches LS1028ARDB.

I first followed the example of the section 4.1.4.2.7 of this documentation: https://www.nxp.com/docs/en/user-guide/REALTIMEEDGEUG_REV2.2.pdf

Contrary to the example you provided in the doc, I would like to let ESA and ESB communicate each other using the FRER (not only ESA towards ESB). Below is my network configuration (the same as the example in the doc).

The first step to allow a communication between ESA and ESB is to set a bridge on each switch (with Linux command ip link and bridge)

On SwitchA we create a bridge br0 :

• frames from swp3 are forwarded to swp2 and swp0
• frames from swp2 are forwarded to swp0 and swp3
• frames from swp0 are frowarded to swp2 and swp3
• swp1 doesn't belong to the bridge to prevent a broadcast storm due to the loop.

On SwitchB we create a bridge br1 where all interfaces belong to it. 

To ensure redundancy when ESA wants to talk to ESB, we use a TSN function (cbgen) which deplicate all the frames from swp0 to swp1.
With this configuration we actually receive two frames on (ESB). It is possible to eliminate the redundant frame with a TSN function (cbrec).

At this step ESA talks to ESB using the redundancy and elimination functions of TSN.

However, ESB cannot communicate with ESA using the redundancy. Remember, swp1 of SwitchA doesn't belong to br0. It means that if the swp0 link is off, swp1 of SwitchA won't be able to transmit the frame to swp3. 

If we set swp1 in forwarding mode (let it belongs to br0) the network will be flooded due to the loop.

My question is the following. It requires specific skills on switches and bridges configuration.

Is it possible set br0 and br1 such as :

• On Switch A :
  • swp3 forward frames only to swp0 and swp2
  • swp2 forward frames only to swp0 and swp3
  • swp0 forward frames only to swp2 and swp3
  • swp1 forward frames only  to swp2 and swp3 when the frames come from ESB (incoming frames)
• On switch B :
  • swp0 forward frames only to swp2 and swp3
  • swp1 forward frames only to swp2 and swp3
  • swp2 forward frames only to swp0 and swp3
  • In other words, prevent swp0 to froward frames to swp1 and reciproquely.

I don't even know if it's possible to do what I would like. Do you have an idea of I can make ESA and ESB communicate each other using the FRER ? I think the key is in the bridge configuration of the two switches.

Thanks a lot for having took the time to read me until here, I stay available if some points wasn't enough clear.

Sincerely,

asanson

[yipingwang]

For your case, you can try the following suggestions:

1. Add swp1 in the bridge, and using "tc filter" to drop some special packets to avoid flooding, like this:
# tc filter add dev swp0 ingress chain 21000 flower skip_sw src_mac 00:04:9f:63:35:21 action drop

about tc filter, you can refer to RM, it need some base configuration:
tc qdisc add dev $ethif clsact
tc filter add dev $ethif ingress chain 0 pref 49152 flower skip_sw action goto chain 10000
tc filter add dev $ethif ingress chain 10000 pref 49152 flower skip_sw action goto chain 11000
tc filter add dev $ethif ingress chain 11000 pref 49152 flower skip_sw action goto chain 12000
tc filter add dev $ethif ingress chain 12000 pref 49152 flower skip_sw action goto chain 20000
tc filter add dev $ethif ingress chain 20000 pref 49152 flower skip_sw action goto chain 21000
tc filter add dev $ethif ingress chain 21000 pref 49152 flower skip_sw action goto chain 30000

2. another way is using vlan tag, you can add swp0-3 into different vlan domain, it can also avoid flooding.
# bridge vlan add dev swp0 vid 101
# bridge vlan add dev swp0 vid 102
you can get guide from real-time-edge RM about vlan usage.

thanks

[asanson]

Hi @yipingwang !

Tanks a lot for your answer. 

I didn't know the existence of "tc filter" command. Thats's so useful !

In my case, I should use it to filter the ingress frame on swp0 and swp1 of Switch A with it's own source MAC address. In the same way, I should filter on ingress of swp0 and swp1 of Switch B the frames with it's own source MAC address.

This configuration shoud prevent a broadcast storm. But how do you handle the broadcast frames (frames with destination MAC address ff:ff:ff:ff:ff:ff) ? I need these frames for the ARP protocol. So it means I can't filter them.

If I was working on L3, I could set a Time To Live (TTL) for all the the broadcast frames. But unfortunately, that's impossible on L2. 

Do you have an idea how can I manage the broadcast frames without flooding the network ?

Thanks.

asanson

[yipingwang]

You can disable broadcast frames for ARP, then set ARP tables manually.

just for reference.

thanks

[asanson]

Thanks for you reply.

Finally, I don't think filtering ingress frame is the best solution to do what I would like to do.

I worked a bit with tc command and I think port mirroring would be a proper solution.

I tried these following commands to mirror all ingress frames on swp1 to swp3 (egress):
# tc qdisc add dev swp1 clsact
# tc fitler add dev swp1 ingress matchall skip_sw action mirred egress mirror dev swp3

And I have this answer from the kernel :
> RTNETLINK answers: Operation not supported
> We have an error talking to the kernel

Is the port mirroring supporting by the LS1028ARDB switch?

I did some more researches and I tried to set port mirroring as described here :
https://medium.com/swlh/traffic-mirroring-with-linux-tc-df4d36116119

After several tests, I noticed that it's impossible to create basic switch object such as a bond, a tunnel or else with ip link command.

For example, the following command returns : 
# ip link add tun0 type gretap remote 10.140.4.88 local 10.140.4.99 dev swp1 
> Error: Unknown device type

It's the same if I want to create a bond :
# ip link add bond0 type bond
> Error: Unknown device type

I think that's note possible to create all these basic switch objects simply because the iplink utility is located in /bin/busybox and this busybox probably don't implement all the ip link functions.

Finally I tried the VLAN solution, but to my mind, It necessary to create a bond with this solution...

@yipingwang could you be more explicit regarding the VLAN tag solution ?

[yipingwang · Accepted solution]

1. About vlan configuration, we have an example for the attached figure, you can refer to the following steps:
a. Set generator switch to be VLAN awareness.
#ip link set name br0 type bridge vlan_filtering 1
b. set swp0, swp1, swp2 in different VLAN on board A.
# bridge vlan add dev swp0 vid 1 pvid
# bridge vlan add dev swp0 vid 2
# bridge vlan add dev swp0 vid 3
# bridge vlan add dev swp1 vid 2 pvid
# bridge vlan add dev swp1 vid 1
# bridge vlan add dev swp2 vid 3 pvid
# bridge vlan add dev swp2 vid 1
c. self-address filtering.
# tc qdisc add dev swp2 clsact
# tc filter add dev swp2 flower skip_sw src_mac 00:01:02:03:04:05 action drop

2. If don't use vlan, you can also disable MAC table auto-learning, then add FDB manually,
a. disable auto-learning:
the related register bit is LEARN_ENA @ ANA_PORT_PORT_CFG, you can refer to the function ocelot_port_set_learning in drivers/net/ethernet/mscc/ocelot.c

b. add FDB:
bridge fdb add 00:01:83:fe:12:01 dev swp1 vlan 1 master static

[HingWong]

Hi yipingwang,

I tried the following command on LS1028RDB, 

# bridge vlan add dev swp0 vid 1 pvid 

however, it says command not found. 

root@TinyLinux:~# bridge vlan add dev swp0 vid 1 pvid
-sh: bridge: command not found

So, what additional software do I need to install to support the bridge command?

[asanson]

Thanks a lot for all your replies @yipingwang 

I think VLAN configuration is the most suitable solution.
I've set it on the on my LS1028ARDB switches and it works as I wanted!

Regarding the tc filter solution, it could have been a good solution but it entails that we have to configure manually the MAC table.
Also I tried to set port mirrorring but unfortunately, it is not supported by the LS1028ARDB switches.

One more time, thanks for the support.

Sincerely, 

asanson