LS1046 Is there a frame statistics tool for MACs

Hmc510 — Wed, 10 May 2023 08:47:07 GMT

I tried using the iperf3 tool to test TCP packet transmission.

find some retransmission in speed 10Mb/s . packet size is 60 byte.(net link is 100Mbps):

the URL: https://community.nxp.com/t5/Layerscape/LS1046-Small-packet-transmission-Network-packet-loss/m-p/1646212#M12416

------------------------------

My understanding is that my PHY No loss occurred. and Qman Fman no packet loss or error was recorded. but QMAN packet is less than PHY's statistics.

Is it possible for the serdes、MACs to be lost ?

Re: LS1046 Is there a frame statistics tool for MACs

stadium_aquino — Thu, 11 May 2023 18:28:27 GMT

https://www.kernel.org/doc/html/latest/networking/statistics.html has a good overview of available ethernet statistics

Re: LS1046 Is there a frame statistics tool for MACs

Hmc510 — Fri, 12 May 2023 01:09:15 GMT

I think it's not supported on LS1046.

How to view PHY and MAC packet receiving information on LS1046?

Re: LS1046 Is there a frame statistics tool for MACs

yipingwang — Tue, 16 May 2023 02:57:30 GMT

You should focus on the RX port counters and the RX MAC counters. You will understand by reading the explanations, the difference between port and MAC.

"For debugging I will put here the MAC base addresses.
The mappings within FMD (FMAN kernel driver) can be found here: https://community.nxp.com/t5/Layerscape-Knowledge-Base/LS1043ARDB-Ethernet-and-FMC-port-mapping/ta-p/1115640

– FM1 mEMAC1: 1AE_0000h
– FM1 mEMAC2: 1AE_2000h
– FM1 mEMAC3: 1AE_4000h
– FM1 mEMAC4: 1AE_6000h
– FM1 mEMAC5: 1AE_8000h
– FM1 mEMAC6: 1AE_A000h
– FM1 mEMAC9: 1AF_0000h
– FM1 mEMAC10: 1AF_2000h

For example: MAC5 is mapped with fm0-port-rx4. This device string is used for port configuration
when fmc tool is executed. (fmc is a tool that permits to apply advanced offload functionallity on
FMAN ports. The functionallity is encoded into a model called PCD - parse classify distribute - embedded
in an xml)
In your debugging you will not used this -fmc I mean. I wanted to explain what fm0-port-rx means in the explanations
that will follow.

(Hint: MAC - port mapping can be found in the decompiled device tree)

The life of a packet for a LS1046 FMAN port is:

RX(at each level there are counters):
Rx MAC -> Port BMI -> FPM -> BMI prepare to enqueue->QMI -> BMI release internal resources -> CPU

TX(at each level there are counters):
BMI - > QMI (dequeue request from FMAN) - > BMI (DMA transaction from DDR to Internal memory) -> TX MAC -> QMI -> BMI (release resources = EBD)

An example of quick dumping of FMAN counters from sysfs(these represent FMBM_RFRC):

find /sys/devices/ -name 'port_frame' -exec cat {} \;
fm0-port-rx5 counter: 0
fm0-port-tx4 counter: 0
FM Port not configured...
FM Port not configured...
fm0-port-tx0 counter: 1365
FM Port not configured...
FM Port not configured...
FM Port not configured...
FM Port not configured...
fm0-port-tx7 counter: 0
fm0-port-rx4 counter: 47279872
FM Port not configured...
fm0-port-rx0 counter: 6800
fm0-port-tx5 counter: 43045909
fm0-port-oh1 counter: 0
FM Port not configured...
fm0-port-rx7 counter: 0
FM Port not configured...
FM Port not configured...
FM Port not configured...
FM Port not configured...
FM Port not configured...

Counters (detailed):

On Rx
0x204 FMBM_RFRC—Rx Frame Counter R/W All zeros 5.5.4.5.26/5-133
0x208 FMBM_RBFC—Rx Bad Frames Counter R/W All zeros 5.5.4.5.27/5-133
0x20C FMBM_RLFC—Rx Large Frames Counter R/W All zeros 5.5.4.5.28/5-134
0x210 FMBM_RFFC—Rx Filter Frames Counter R/W All zeros 5.5.4.5.29/5-135
0x214 FMBM_RFDC—Rx Frame Discard Counter R/W All zeros 5.5.4.5.30/5-135
0x218 FMBM_RFLDEC—Rx Frames List DMA Error Counter R/W All zeros 5.5.4.5.31/5-136
0x21C FMBM_RODC—Rx Out of Buffers Discard Counter R/W All zeros 5.5.4.5.32/5-137
0x220 FMBM_RBDC—Rx Buffers Deallocate Counter R/W All zeros 5.5.4.5.33/5-138
0x224 FMBM_RPEC—RX Prepare to enqueue Counter. R/W All zeros 5.5.4.5.34/5-138

cat /sys/bus/platform/devices/soc/1a00000.fman/1a8a000.port/fm_port_bmi_regs | grep -i rfr
#FMBM_RBFC is not exported. We use a tool such as iomem or devmem
./iomem r32be:2 0x1a8a208
cat /sys/bus/platform/devices/soc/1a00000.fman/1a8a000.port/fm_port_bmi_regs | grep -i rlf
cat /sys/bus/platform/devices/soc/1a00000.fman/1a8a000.port/fm_port_bmi_regs | grep -i rff
./iomem r32be:2 0x1a8a214
cat /sys/bus/platform/devices/soc/1a00000.fman/1a8a000.port/fm_port_bmi_regs | grep -i dec
cat /sys/bus/platform/devices/soc/1a00000.fman/1a8a000.port/fm_port_bmi_regs | grep -i rodc
./iomem r32be:2 0x1a8a220
./iomem r32be:2 0x1a8a224

On Tx
0x204 FMBM_TFRC—Tx Frame Counter R/W All zeros 5.5.4.6.19/5-163
0x208 FMBM_TFDC—Tx Frames Discard Counter R/W All zeros 5.5.4.6.20/5-163
0x20C FMBM_TFLEDC—Tx Frames Length Error Discard Counter
0x210 FMBM_TFUFDC—Tx Frames Unsupported Format Discard Counter
0x214 FMBM_TBDC—Tx Buffers Deallocate Counter R/W All zeros 5.5.4.6.23/5-165

Check:
cat /sys/bus/platform/devices/soc/1a00000.fman/1aaa000.port/fm_port_bmi_regs | grep -i tfr
cat /sys/bus/platform/devices/soc/1a00000.fman/1aaa000.port/fm_port_bmi_regs | grep -i tfdc
cat /sys/bus/platform/devices/soc/1a00000.fman/1aaa000.port/fm_port_bmi_regs | grep -i tfl
cat /sys/bus/platform/devices/soc/1a00000.fman/1aaa000.port/fm_port_bmi_regs | grep -i tfuf

#we do not have this register mapped so we read it using devmem or iomem
./iomem r32be:2 0x1aab214

On Rx
0x420 FMQM_PnETFC—PortID n Enqueue Total Frame Counter R/W 0x0000_0000 5.6.3.2.20/5-266

Check:
cat /sys/bus/platform/devices/soc/1a00000.fman/1a8a000.port/fm_port_qmi_regs | grep -i pnet

On TX
0x434 FMQM_PnDTFC—PortID n Dequeue Total Frame Counter R/W 0x0000_0000 5.6.3.2.24/5-270
0x43C FMQM_PnDCC—PortID n Dequeue Confirm Counter R/W 0x0000_0000 5.6.3.2.26/5-270

Check:
cat /sys/bus/platform/devices/soc/1a00000.fman/1aaa000.port/fm_port_qmi_regs | grep -i dtfc
cat /sys/bus/platform/devices/soc/1a00000.fman/1aaa000.port/fm_port_qmi_regs | grep -i cc

MAC counters:
They can be found here:
cat /sys/devices/platform/soc/soc:fsl,dpaa/soc:fsl,dpaa:ethernet@2/net/eth2/mac_tx_stats
cat /sys/devices/platform/soc/soc:fsl,dpaa/soc:fsl,dpaa:ethernet@2/net/eth2/mac_rx_stats

Note in case the sysfs entries differ from above, identify the counters by:

find / -name "mac_rx_stats"

You can then consult the manual for offsets that you are interested in.
If the frames have reached on RX and you see counters incremented on BMI / QMI that means that everything has
passed succesfully through MAC RX.

On the TX, if there are no error counters on MAC incremented that means that everything has passed succesfully
through QMI /BMI"

Observation:

If there are problems at SERDES level then I would expect for the PCS status register to indicate link down (link status is accurate indicator that valid symbols are being received) .If link is down then you should experience sporadic traffic stoppage and also PCS link flap.
Another indicator for invalid bits in SERDES is the CDR_LCK. It will be 0 in case of errors. (you will lose the CDR lock)
CDR_LCK can be checked in the SERDES registers.

If packets are corrupted by SERDES, you should either see errors in the MAC error counters or you should see a difference in counters between the MAC and the PORT.

Re: LS1046 Is there a frame statistics tool for MACs

Hmc510 — Tue, 16 May 2023 06:10:54 GMT

Thank you for your detailed response. Thank you very much.

I have understood your introduction.

There is a question, where can I find the descriptions of these MAC registers?

Re: LS1046 Is there a frame statistics tool for MACs

Hmc510 — Wed, 17 May 2023 10:02:48 GMT

Hi I tried recording some data. it lost 23 packet this time.

I couldn't find a debugging method for serdes.

Do you have any suggestions for these data? Is it a packet loss caused by the serdes.

iperf3 retr=23
PC(TX:)17120
YT8614: PHY = 17120
MAC: rpkt_l=17097
port: fmbm_rfrc = 17097
RX (netstat -i) = 17097

Re: LS1046 Is there a frame statistics tool for MACs

Hmc510 — Fri, 19 May 2023 02:22:35 GMT

This issue is also reproduced on the LS1043 board. I don't understand how this is caused

topic Re: LS1046 Is there a frame statistics tool for MACs in Layerscape

LS1046 Is there a frame statistics tool for MACs

Re: LS1046 Is there a frame statistics tool for MACs

Re: LS1046 Is there a frame statistics tool for MACs

Re: LS1046 Is there a frame statistics tool for MACs

Re: LS1046 Is there a frame statistics tool for MACs

Re: LS1046 Is there a frame statistics tool for MACs

Re: LS1046 Is there a frame statistics tool for MACs