2402005_en-US

[#1]

How to Test TX PAUSE Frames with DPDK enetc4 VF Driver on i.MX95 / i.MX9x Series

This post walks through end-to-end steps to enable and verify TX PAUSE frame (IEEE 802.3x flow control) generation on the i.MX95 / i.MX9x series using DPDK 25.11 with the enetc4 VF driver and a Spirent traffic generator. I also share an optional debug patch that lowers the PAUSE trigger threshold for quick lab reproduction and adds register readback prints to dmesg.

---

Overview
--------

On i.MX95 the ENETC4 Ethernet controller uses a PF/VF split:

- The kernel PF driver (fsl_enetc4, Linux 6.18+) owns the MAC, PHY negotiation, and PAUSE configuration.
- The DPDK VF driver (net/enetc, DPDK 25.11) owns the receive rings in the DPDK application.

When the link partner negotiates PAUSE, the kernel PF configures the MAC and notifies the DPDK VF via a mailbox message. The VF then enables congestion signaling on its Rx rings. When incoming traffic fills those rings past the configured threshold, the hardware automatically emits PAUSE frames toward the sender.

The trigger chain looks like this:

High-rate ingress traffic fills VF Rx rings
--> ICM fill level crosses PPAUONTR threshold
--> MAC emits IEEE 802.3x PAUSE frame to link partner
--> link partner pauses its transmitter

---

Hardware Setup
--------------

- i.MX95 EVK (or any i.MX9x board with ENETC4)
- 10G SFP+ DAC cable or fiber between i.MX95 ENETC4 port and Spirent TestCenter port
- Spirent TestCenter (or equivalent traffic generator with flow control capture)

Software versions used in this guide:

- Kernel: Linux 6.18+ with fsl_enetc4 PF driver
- DPDK: 25.11 (net/enetc VF PMD)
- ethtool: 6.x

---

Step 1 — Enable PAUSE on the Kernel PF Interface
-------------------------------------------------

The kernel PF interface (typically eth1 for ENETC4 port 1) must have TX PAUSE enabled before the link comes up so that phylink can negotiate it with the link partner.

# Identify the kernel PF interface
ip link show | grep -E "eth[0-9]"

# Enable TX and RX PAUSE (autoneg lets the link partner also advertise PAUSE, it is off since we are using spirent)
ethtool -A eth1 tx on rx on autoneg off

# Bring the link up
ip link set eth1 up

# After link is up, verify PAUSE was negotiated
ethtool -a eth1

Expected output:

Pause parameters for eth1:
Autonegotiate: off
RX: on
TX: on

If TX shows "off" after link up, the link partner may not have advertised PAUSE capability. Try forcing it:

ethtool -A eth1 tx on rx on autoneg off

---

Step 2 — Verify ethtool Statistics Are Available
-------------------------------------------------

Confirm that the ethtool stats interface is working before starting traffic:

ethtool -S eth1 | grep -E "txpf|rxpf|pause"

You should see counters like txpf_frames and rxpf_frames (both 0 at this point). If you see "no stats available", verify your kernel build includes the ethtool ops for enetc4.

---

Step 3 — Bind the DPDK VF to igb_uio
-----------------------------------

note bootargs: must have iommu_passthrough=1

# Load the VFIO driver
modprobe igb_uio

echo 1 > /sys/bus/pci/devices/0002\:00\:10.0/sriov_numvfs
echo igb_uio > /sys/bus/pci/devices/0002\:00\:12.0/driver_override
echo 0002:00:12.0 > /sys/bus/pci/drivers/fsl_enetc_vf/unbind
echo 0002:00:12.0 > /sys/bus/pci/drivers/igb_uio/bind
ip link set eth1 vf 0 trust on

---

Step 4 — Allocate Hugepages
-----------------------------

# 4 x 1 GB hugepages (recommended for 10G line-rate testing)
echo 4 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
mount -t hugetlbfs none /dev/hugepages

# Verify allocation
grep HugePages /proc/meminfo
# HugePages_Total: 4
# HugePages_Free: 4

---

Step 5 — Start testpmd
------------------------

For PAUSE testing the goal is to build up backpressure in the VF Rx rings so the ICM congestion threshold is crossed. Use rxonly mode with a small Rx descriptor count so the ring fills up quickly under load.

testpmd \
-l 0-3 -n 4 \
-a 0002:00:12.0 \
-- \
--rxd=256 \
--txd=512 \
--nb-cores=2 \
--rxq=1 --txq=1 \
--forward-mode=rxonly \
--stats-period=5

Inside the testpmd prompt:

testpmd> set fwd rxonly
testpmd> start

testpmd is now receiving and the DPDK VF Rx rings will fill under high-rate ingress traffic, generating the congestion signal that drives PAUSE frame emission.

---

check stats via:
ethtool --include-statistics -a eth1

Step 6 — Configure Spirent TestCenter
---------------------------------------

Spirent port sending TO i.MX95 (ingress traffic):

- Frame size: 64 bytes or smaller (smaller frames fill rings faster)
- Rate: 100% line rate (10 Gbps)
- Frame type: Ethernet II / IPv4
- Destination MAC: MAC address of the i.MX95 ENETC4 VF interface
- Destination IP: IP address assigned to the i.MX95 interface

Spirent port receiving FROM i.MX95 (PAUSE capture):

- Port Properties → Flow Control → Enable IEEE 802.3x PAUSE
- Enable Capture → Filter EtherType: 0x8808
- Results → Port Results → watch "Flow Control Frames Received"
This counter increments every time a PAUSE frame arrives from i.MX95.
- Optional: Results → Flow Analysis → check "Pause Duration (quanta)"
Expected value: ~32767 (0x7FFF)

Alternative capture without Spirent license:

# On any PC with a tap on the wire
tcpdump -i eth0 'ether proto 0x8808' -v
# PAUSE frame: dst 01:80:c2:00:00:01, EtherType 0x8808, opcode 0x0001

---

Step 7 — Verify PAUSE Frames Are Being Sent
---------------------------------------------

Start Spirent at 100% line rate, then on the i.MX95 host run:

watch -n 1 'ethtool -S eth1 | grep -E "txpf|rxpf"'

Expected output when PAUSE is active:

txpf_frames: <-- TX PAUSE frames sent by i.MX95

On Spirent, "Flow Control Frames Received" should be incrementing at the same time.

---

Optional — Debug Patch for Faster Lab Reproduction
----------------------------------------------------

I am attaching a patch file to this post. This useful for bring-up and debug — this is not required for production use.

Adds dev_info() prints to enetc4_set_tx_pause() and enetc_set_congestion_mode()
in the kernel driver. After link-up you will see in dmesg:

fsl_enetc4 0000:00:00.0: enetc4_set_tx_pause: tx_pause=1
fsl_enetc4 0000:00:00.0: PPAUONTR = 0x00001000
fsl_enetc4 0000:00:00.0: PPAUOFFTR = 0x00000400
fsl_enetc4 0000:00:00.0: PM_CMD_CFG(0) = 0x000000c3 TX_EN=1 RX_EN=1 TXP=0
fsl_enetc4 0000:00:00.0: set_congestion_mode: enable=1 num_rx_rings=1
fsl_enetc4 0000:00:00.0: ring[0] readback rbmr=0x00000010

This confirms the kernel PF configured the MAC and set the RBMR congestion mode bit
on the PF rings. The DPDK VF should receive the same setting via mailbox.

also

Lowers the PAUSE trigger threshold to PPAUONTR=4096 bytes so that PAUSE frames
are generated at much lower traffic rates — useful for quick lab tests without
needing a full 10G line-rate traffic generator. Also adds ICM register definitions
(PRXBCR, PRXBCHWMR) so you can observe the ICM fill level via ethtool -S.

After applying this patch, check ICM fill level during traffic:

ethtool -S eth1 | grep -E "prxbcr|prxbchwmr"
# prxbcr_bytes:
# prxbchwmr_bytes:

When prxbchwmr_bytes >= 4096, the threshold has been crossed and PAUSE should fire.

---

Key Register Reference (ENETC4)
---------------------------------

Note: ENETC4 register offsets start at 0x5000 for MAC/PM registers.
This is different from ENETC v1 which uses 0x8000. Use ethtool -S for
PM counters since PF BAR0 is IOMMU-protected on i.MX95.

Register Offset Description
PM_CMD_CFG(0) 0x5008 MAC config register (TXP = BIT 15)
PM_TXPF(0) 0x5218 TX PAUSE frames sent (64-bit)
PM_RXPF(0) 0x5118 RX PAUSE frames received (64-bit)
PPAUONTR 0x108 ICM fill level threshold to START PAUSE
PPAUOFFTR 0x10C ICM fill level threshold to STOP PAUSE
PRXBCR 0x128 Current ICM RX fill level (live, read-only)
PRXBCHWMR 0x12C ICM RX peak fill since boot (read-only)

For VF Rx ring registers, VF BAR0 is accessible via devmem2:

# Check RBMR of VF ring 0 — BIT 4 = CM (congestion mode)
# Replace VF_BAR0 with your actual address (find via /sys/bus/pci/devices/.../resource0)
devmem2 w
# Expected when PAUSE is active: 0x00000010

---

Environment
-----------

SoC: i.MX95, i.MX943 (i.MX9x series with ENETC4)
Kernel: Linux 6.18+ (fsl_enetc4 PF driver)
DPDK: 25.11 (net/enetc VF PMD)
Tool: testpmd, ethtool 6.x, Spirent TestCenter

Hope this helps. Happy to answer questions on the setup.