topic CEETM configuration in Layerscape

CEETM configuration

LeNormich — Tue, 16 Mar 2021 15:52:16 GMT

Hello,

I'm facing issues using CEETM on a LX2160ARDB.

My setup is the following:

| ------- | |
Traffic Generator | ------- | LX2160 | ------- | capture

generator sends frames on 2 interfaces of LX2160 with ip dst = 192.85.3.3 and ip src=192.85.x.3 (x= 1,2)

I try to aggregate these flows to 1 output interface and use ceetm to shape traffic.

I use the following commands to configure my board:

ls-addni dpmac.18 //creates eth2 ls-addni dpmac.3 //creates eth3 ls-addni -nq=16 -t=8 dpmac.4 //creates eth4 ifconfig eth2 hw ether 00:e0:0c:77:01:00 ifconfig eth2 192.85.1.1/24 up ifconfig eth3 hw ether 00:e0:0c:77:02:00 ifconfig eth3 192.85.2.1/24 up ifconfig eth4 hw ether 00:e0:0c:77:03:00 ifconfig eth4 192.85.3.1/24 up arp -s 192.85.3.3 00:94:10:00:00:03 iptables -P FORWARD ACCEPT // at that point, frames are transmitted and present in capture //next ceetm configuration tc qdisc add dev eth4 root handle 1: ceetm type root tc class add dev eth4 parent 1: classid 1:1 ceetm type root cir 1000mibit tc qdisc add dev eth4 parent 1:1 handle 2: ceetm type prio prioA 3 prioB 1 separate 1 tc class add dev eth4 parent 2: classid 2:1 ceetm type prio mode WEIGHTED_A weight 100 tc class add dev eth4 parent 2: classid 2:2 ceetm type prio mode WEIGHTED_A weight 200 tc filter add dev eth4 parent 1: protocol ip u32 match ip src 192.85.1.3/32 flowid 1:1 tc filter add dev eth4 parent 2: protocol ip u32 match ip src 192.85.1.3/32 flowid 2:1 tc filter add dev eth4 parent 1: protocol ip u32 match ip src 192.85.2.3/32 flowid 1:1 tc filter add dev eth4 parent 2: protocol ip u32 match ip src 192.85.2.3/32 flowid 2:2

The issue is that only one flow is transmitted: when doing "tc -s class show eth4" flowid 2:1 frames are all discarded, only flow 2:2 is transmitted (even though link is not full and both are completely transmitted when i remove ceetm)

Is there a problem with my configuration?

Thanks,

Best regards,

Re: CEETM configuration

yipingwang — Wed, 17 Mar 2021 07:42:47 GMT

Please check whether the following Kernel configurations have been enabled.

CONFIG_FSL_DPAA2_ETH_CEETM=y
CONFIG_NET_SCHED=y

CONFIG_NET_CLS=y
CONFIG_NET_CLS_ACT=y
CONFIG_NET_CLS_U32=y
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_U32=y

Please try the following example.

$ tc qdisc add dev eth4 root handle 1: ceetm type root

$ tc class add dev eth4 parent 1: classid 1:1 ceetm type root cir 1000mibit

$ tc qdisc add dev eth4 parent 1:1 handle 2: ceetm type prio prioA 3 prioB 1 separate 1

$ tc class add dev eth4 parent 2: classid 2:1 ceetm type prio mode STRICT_PRIORITY

$ tc class add dev eth4 parent 2: classid 2:2 ceetm type prio mode WEIGHTED_A weight 200

$ tc filter add dev eth4 parent 1: protocol ip u32 match ip src 192.85.1.3/32 flowid 1:1

$ tc filter add dev eth4 parent 2: protocol ip u32 match ip src 192.85.1.3/32 flowid 2:1

$ tc filter add dev eth4 parent 1: protocol ip u32 match ip src 192.85.2.3/32 flowid 1:1

$ tc filter add dev eth4 parent 2: protocol ip u32 match ip src 192.85.2.3/32 flowid 2:2

Re: CEETM configuration

LeNormich — Wed, 17 Mar 2021 15:01:10 GMT

Hello,

II just tried it with the same result: only traffic from 192.85.2.3 is going through (traffic flow 2:2)

Re: CEETM configuration

LeNormich — Thu, 18 Mar 2021 17:30:14 GMT

When trying with more flows, I always observe that traffic flow 2:1 is systematically rejected.

Any idea as to why this happens?

Thanks

Re: CEETM configuration

yipingwang — Mon, 22 Mar 2021 06:12:21 GMT

I have made a setup a little bit different from you (but this does not affect the ceetm) due to the limited boardfarm configuration.

In my setup one port - MAC17 is not connected to TC but to another LX2 that injects traffic into MAC17, while MAC3 and MAC4 are linked with the TC.

Two flows are injected in parallel:
-one into M17 using tcpreplay
-one into MAC3 from the TC

Flows are forwarded to MAC4 and shaped according to the weights.
In the attached caption it can be seen the different bandwidth for each received flow:
- for 192.85.1.3: 141Mbps
- for 192.168.2.3: 824Mbps

The output of tc shows no rejections (ignore the values, because they are not in synch with what it is in TC. I have tested on MAC17 first with a ping flood, that's why the difference was so huge in the beginning. The idea is to observe that there is no rejection as you suggested):

tc -s class show dev eth3
class ceetm 1:1 parent 1: leaf 2: type root CIR 1048Mbit EIR 0bit CBS 0 EBS 0 coupled 0

class ceetm 2:1 parent 2: type prio mode WEIGHTED_A weight 100

ceetm:
deq bytes 3228190
deq frames 32941
rej bytes 0
rej frames 0

class ceetm 2:2 parent 2: type prio mode WEIGHTED_A weight 200

ceetm:
deq bytes 42951208872
deq frames 33660822
rej bytes 0
rej frames 0

Below are the commands I have execued for the setup. In my case, I used an empty dpl that's why I had to create the macs and dpnis.
There is no issue with the ceetm config according to what I observe. you need to check your setup.

My commands:

restool dpmac create --mac-id=3
restool dpmac create --mac-id=4
restool dpmac create --mac-id=17
restool dprc assign dprc.1 --object=dpmac.3 --plugged=1
restool dprc assign dprc.1 --object=dpmac.4 --plugged=1
restool dprc assign dprc.1 --object=dpmac.17 --plugged=1
ls-addni dpmac.17
ls-addni dpmac.3
ls-addni -nq=16 -t=8 dpmac.4

ifconfig eth1 hw ether 00:e0:0c:77:01:00
ifconfig eth1 192.85.1.1/24 up
ifconfig eth2 hw ether 00:e0:0c:77:02:00
ifconfig eth2 192.85.2.1/24 up
ifconfig eth3 hw ether 00:e0:0c:77:03:00
ifconfig eth3 192.85.3.1/24 up
arp -s 192.85.3.3 00:94:10:00:00:03

echo 1 > /proc/sys/net/ipv4/ip_forward

tc qdisc add dev eth3 root handle 1: ceetm type root
tc class add dev eth3 parent 1: classid 1:1 ceetm type root cir 1000mibit
tc qdisc add dev eth3 parent 1:1 handle 2: ceetm type prio prioA 3 prioB 1 separate 1
tc class add dev eth3 parent 2: classid 2:1 ceetm type prio mode WEIGHTED_A weight 100
tc class add dev eth3 parent 2: classid 2:2 ceetm type prio mode WEIGHTED_A weight 200
tc filter add dev eth3 parent 1: protocol ip u32 match ip src 192.85.1.3/32 flowid 1:1
tc filter add dev eth3 parent 2: protocol ip u32 match ip src 192.85.1.3/32 flowid 2:1
tc filter add dev eth3 parent 1: protocol ip u32 match ip src 192.85.2.3/32 flowid 1:1
tc filter add dev eth3 parent 2: protocol ip u32 match ip src 192.85.2.3/32 flowid 2:2