how to setup mscc_felix switch driver by user api on LS1028A board

xuchegndi · ‎09-10-2021

Hi All, I am using LS1028A board,

how can I drive the felix_switch_ops in user space?

can I user some tools?

I found that iproute2/brctl/ip link may not work in hardware switch

Thanks so much!

xuchegndi · ‎09-22-2021

mark

yipingwang · ‎09-12-2021

These tools iproute2/brctl/ip link should all work.

You need to configure all the ports of the switch up before using these tools.

Please refer to the following script to configure switch.

#!/bin/bash
#
# Simple switch configuration without bridge
# Assume both ENETC and Felix drivers are already loaded

MAC_ROOT=bc:8d:bf:7c:5b
swpip=100.1

# bring up ENETC switch port
seth=$(ls /sys/bus/pci/devices/0000:00:00.2/net/)
ip link set $seth up

# Configure switch ports
swps=($(ls /sys/bus/pci/devices/0000:00:00.5/net/))
let nr=${#swps[@]}
for (( i=0; i<$nr; i++ ))
do
#ip link set ${swps[$i]} address $MAC_ROOT:$(echo "${swps[$i]}" | tr -dc '0-9')
#ip addr add ${swpip}.${i}.1/24 dev ${swps[$i]}
ip link set ${swps[$i]} up
done

xuchegndi · ‎09-13-2021

thanks so much!

I 've made that configuration in fact.

I don't know the interaction between driver and iproute2. for example,

in iproute2/bridge/fdb.c, fdb_show() function will call netlink communication,

and in kernel ocelot/felix.c, felix_fdb_dump() uses a callback function.

how do they interaction?

we may found a bug about fdb_show in OpenIL, hardware-base learning certainly learned the entry mac=ea,40,78,ac,8a,69,vid=1,port=1

it is strange that when using "bridge fdb show br br0" (assume that we have add swp0-3 to bridge br0 and set them up ) ,we can not see mac=ea,40,78,ac,8a,69,vid=1,port=1 in user space print, but we do see the entry in kernel debug by using printk! what is more, the switch can transfer the frame with dest mac ea,40,78,ac,8a,69, which means that switch surely learned the mac.

I am trying to figure out the bug by modifying codes in kernel and iproute2. but I do not know how to debug from iproute2 --> kernel if iproute2 is used by netlink.

I am waiting for your reply, thanks so much.

yipingwang · ‎09-14-2021

I review the related code, the call relation is the following:

net/core/rtnetlink.c:

rtnl_register(PF_BRIDGE, RTM_GETNEIGH, rtnl_fdb_get, rtnl_fdb_dump, 0);

rtnl_fdb_dump -> ndo_fdb_dump(dsa_slave_fdb_dump)

net/dsa/slave.c: net/dsa/port.c:

dsa_slave_fdb_dump -> dsa_port_fdb_dump -> port_fdb_dump(felix_fdb_dump)

drivers/net/dsa/ocelot/felix.c:

felix_fdb_dump -> ocelot_fdb_dump -> ocelot_mact_read:

ret = cb(entry.mac, entry.vid, is_static, data);

drivers/net/ethernet/mscc/ocelot.c:

ocelot_mact_read will loop through all the mac tables entries.

cb is dsa_slave_port_fdb_do_dump in net/dsa/slave.c

Which version of Openil is using?

Can you post the output in kernel debug by using printk?

xuchegndi · ‎09-15-2021

Thanks for your reply!

1. we are using Openil version 1.10

2. we would like to show our modified code here, fdb dump code in driver/ethernet/mscc/ocelot.c:

int ocelot_fdb_dump(struct ocelot *ocelot, int port,
dsa_fdb_dump_cb_t *cb, void *data)
{
int i, j;

/* Loop through all the mac tables entries. */
for (i = 0; i < ocelot->num_mact_rows; i++) {
for (j = 0; j < 4; j++) {
struct ocelot_mact_entry entry;
bool is_static;
int dst = 0, ret, dst_port = 0;

ret = ocelot_mact_read(ocelot, i, j, &dst, &entry);

//Entry is NOT valid, continue the loop. Modified by shaoguangyuan.
if (ret == -EINVAL)
continue; // original code is :return; # this cause when meeting one invalid entry(maybe the enety ageing time is out), code will quit the loop, but we want to continue the valid entry.
if (ret)
return ret;

//we add this code here, for filtering by port. because when swp0~swp3, port=1,2,3,4; but dst_port=1,2,4,8, the dst_port is a mask.
dst_port = ocelot_read_rix(ocelot, ANA_PGID_PGID, dst);
if (!(dst_port & BIT(port)))
continue;

//we add debug info here
dev_err(ocelot->dev, "MAC = %02x:%02x:%02x:%02x:%02x:%02x\n",
entry.mac[0], entry.mac[1], entry.mac[2], entry.mac[3], entry.mac[4], entry.mac[5]);
dev_err(ocelot->dev, "VID = %d, port = %d\n", entry.vid, port);
dev_err(ocelot->dev, "\n");

is_static = (entry.type == ENTRYTYPE_LOCKED);

ret = cb(entry.mac, entry.vid, is_static, data);
if (ret)
return ret;
}
}

3. the debug mesg as follows.

root@someone:~# bridge fdb show br switch
[ 9886.575567] 000: mscc_felix 0000:00:00.5: ****Debug Mesg For fdb dump****
[ 9886.600956] 000: mscc_felix 0000:00:00.5: MAC = 11:22:33:44:55:66
[ 9886.600962] 000: mscc_felix 0000:00:00.5: VID = 1, port = 0
[ 9886.600965] 000: mscc_felix 0000:00:00.5:
[ 9886.612475] 000: mscc_felix 0000:00:00.5: ****Debug Mesg For fdb dump****
[ 9886.647488] 000: mscc_felix 0000:00:00.5: ****Debug Mesg For fdb dump****
[ 9886.682433] 000: mscc_felix 0000:00:00.5: ****Debug Mesg For fdb dump****
11:22:33:44:55:66 dev swp0 vlan 1 offload master switch permanent
11:22:33:44:55:66 dev swp0 offload master switch permanent
82:b4:18:0a:0a:e6 dev swp0 vlan 1 master switch permanent
82:b4:18:0a:0a:e6 dev swp0 master switch permanent
11:22:33:44:55:66 dev swp0 vlan 1 self static
33:33:00:00:00:01 dev switch self permanent
01:00:5e:00:00:6a dev switch self permanent
33:33:00:00:00:6a dev switch self permanent
01:00:5e:00:00:01 dev switch self permanent
33:33:ff:0a:0a:e6 dev switch self permanent
33:33:00:00:00:fb dev switch self permanent

we can see that only the entry added to table by hand will show in printk

but in console-print, we can see many entries, I think ,some entries may be cpu-soft-table, which I do not want to show. after some tests, I conclude that

1. the printk entries are in the hardware-table;

2. the console-print entries by "bridge fdb show br switch" include hardware-table and cpu-soft-table.

My purpose is to get hardware-table only. but how can I filter all the hardware-table? I suggest that:

a. the console-print entries with "dev=swp0~swp3 and flags=self" belongs to hardware-table

b. others are cpu-soft-table

my finally questions are:

1. are my conclusions right, if not ,can you mark for me ?

2. how can i get console-print entries only?

3. what is the flags "master" for in cmd bridge? if I add one entry to switch , do I need to add flags master? like bridge fdb add dev swp0 11:22:33:44:55:66 master

yipingwang · ‎09-17-2021

I have tested the case in ls1028ardb board and Openil V1.10, we recommend to use "bridge fdb show br switch dev swp0" to show, and I got the following output:

[root@LS1028ARDB ~] # bridge fdb show br switch dev swp0

11:22:33:44:55:66 vlan 1 offload master switch static 92:a1:5d:d8:36:6c master switch

5e:37:66:00:bb:a8 vlan 1 master switch permanent

5e:37:66:00:bb:a8 master switch permanent

"11:22:33:44:55:66" is added manually, so its entry has keyword "static".

the recommended command is "bridge fdb add dev swp0 11:22:33:44:55:66 vlan 1 master static".

"92:a1:5d:d8:36:6c" is that learned by switch, the entry has "master switch", but don't have keyword "permanent".

"5e:37:66:00:bb:a8" is mac address of switch, so it has keyword "permanent".

so you can distinguish different types of entries.

I think your patch is valid to fix a bug in mscc switch driver.

besides, I put your patch to Openil V1.11, I got the following output,

[root@LS1028ARDB ~] # bridge fdb show br switch dev swp0 [ 297.704091] 000: mscc_felix 0000:00:00.5: MAC-0 = a6:21:11:3e:4b:42 [ 297.704100] 000: mscc_felix 0000:00:00.5: VID = 1, port = 0 [ 297.704104] 000: mscc_felix 0000:00:00.5:

[ 297.711182] 000: mscc_felix 0000:00:00.5: MAC-1 = 34:48:ed:4d:65:62 [ 297.711188] 000: mscc_felix 0000:00:00.5: VID = 1, port = 0 [ 297.711191] 000: mscc_felix 0000:00:00.5:

[ 297.712686] 000: mscc_felix 0000:00:00.5: MAC-2 = 11:22:33:44:55:66 [ 297.712691] 000: mscc_felix 0000:00:00.5: VID = 1, port = 0 11:22:33:44:55:6[ 297.712694] 000: mscc_felix 0000:00:00.5:

6 vlan 1 offload master switch static

96:fc:74:f5:f5:6c vlan 1 master switch permanent 96:fc:74:f5:f5:6c master switch permanent

a6:21:11:3e:4b:42 vlan 1 self

34:48:ed:4d:65:62 vlan 1 self

11:22:33:44:55:66 vlan 1 self static

the lasted three entry is that you want to get.

the output is seemly clear than V1.10, so you can also try Openil V1.11.

xuchegndi · ‎09-23-2021

Thanks so much.

How can I get code on OpenIL-V1.11? I do not find in https://github.com/openil/openil in any branch or tag.

Can you give me some guidance about using your patch?

-----------------------------

By the way, could you please give me some guidance of reading certain register's value in shell, for example the register ES_REW_PORT_TAG_CFG

I am waiting for your reply!

yipingwang · ‎09-23-2021

you can get Openil V1.11 from NXP website,

about how to read register, It's a little diffcult to get base address of registers, so I recommend to RW regiter from kernel driver.

take ES_ANA_PORT_QOS_CFG as example,

My Reference Manual is "QorIQ LS1028A Reference Manual, Rev. 0, 12/2019",

MSCC base address is 0x1fc000000,

from "Table 254. List of Targets and Base Addresses" in Page 1759, we can get ANA base address is 0x280000 in MSCC memory, REW base address is 0x030000,

from "Table 255. Register Groups in ANA" in Page 1760, we can get PORT base address is 0x00007800 in ANA memory, and spacing is 0x100

from "16.4.3.1.7 ES_ANA_PORT memory map" in Page 1764, we can get ES_ANA_PORT_QOS_CFG offset is 0x08,

so ES_ANA_PORT_QOS_CFG is:

0x1fc000000 + 0x280000 + 0x00007800 + port_number * 0x100 + 0x08;

similarly, ES_REW_PORT_TAG_CFG should be:

0x1fc000000 + 0x030000 + 0x00000000 + port_number * 0x80 + 0x04;

you can use "devmem" tool to read or write the related register.

xuchegndi · ‎09-26-2021

Thanks so much! I used the tool devmem, and it works.

Recently,

I found a problem that it seems that the storm policer for CPU port doesnot work.
According to the document LS1028ARM, flooding frames can be policed by the storm policers, inculding frames that are flooded to the CPU port.

As known, a unicast frame whoes DMAC is not learned by the machine will be flooded to all ports except the source port. According to the kernel code, the CPU port is set in the unicast flooding PGID:

To test that, I send unknown unicast frames to the machine, and the CPU is indeed occupied by 100% usage.
To avoide this, I want to use the storm policer to limite the rate of packes flooding to the CPU. So I configered the register STORMLIMIT_CFG like this:

The result of this test tells that this configuration seems not work, because the usage of CPU is still 100%. But when I set the field STORM_MODE of this register to 3(so that both the CPU port and the front port flooding rate will be limited). This time, the result show that the front port flooding rate do limited to 1 frame/sec, but the CPU usage is still almost 100%.
It seems that the storm policer for CPU port does not work. Could you please have a look of this problem? It will be very kind of you if you can give some guidance of how to limit the rate of frames flooding to the CPU port.
Thanks a lot.

yipingwang · ‎09-26-2021

Please create a new thread for your new problem.

xuchegndi · ‎09-27-2021

Hi Wang, I have created a new topic for this problem, could you please have a look of that? Thank you!

Here is the link:

https://community.nxp.com/t5/Other-NXP-Products/Storm-Protection-for-CPU-port-in-board-LS1028ARM/m-p...

yipingwang · ‎09-23-2021

$ git clone https://github.com/openil/openil.git
$ cd openil
# checkout to the 2021.04 v1.11 release
$ git checkout OpenIL-v1.11-202104 -b OpenIL-v1.11-202104