Same address issue of two different packet from DPDK on LX2160A

[TSCC]

Two ethernet port on the board. Port1 for receiving on cpu1 and port2 for transmitting on cpu2.
Received packets will be enqueued to a ring buffer. Other CPUs(3-15) dequeue packets and do some operations.

The problem is the mbuf address will be replaced to a new packet by the next calling of rte_eth_rx_burst.
If the mbuf still in the ring buffer, then there is a chance that the mbuf is replaced to a wrong packet.

I have used rte_pktmbuf_refcnt_update function to prevent the mbuf will not be released.

SDK version : LSDK-21.08
DPDK git tag: LSDK-21.08

Ethernet port init log:

EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Detected shared linkage of DPDK
root@localhost:/tmp# EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
fslmc: Skipping invalid device (power)
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-2048kB
EAL: No available hugepages reported in hugepages-32768kB
EAL: No available hugepages reported in hugepages-64kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: dpni.1: netdev created
PMD: dpni.2: netdev created
EAL: No legacy callbacks, legacy socket not created
Port: Name IfIndex Alias NUMA PCI
0: net_dpaa2 0 0
1: net_dpaa2 0 0

=== port to lcore mapping table (# lcores 3) ===
lcore: 0 1 2 Total
port 0: ( T) ( 1: 1) ( 0: 0) = ( 1: 1)
port 1: ( T) ( 0: 0) ( 1: 1) = ( 1: 1)
Total : ( 0: 0) ( 1: 1) ( 1: 1)
Display and Timer on lcore 0, rx:tx counts per port/lcore

>>>> Configuring 2 ports, MBUF Size 1618, MBUF Cache Size 4096
Lcore:
1, RX-TX
RX_cnt( 1): (pid= 0:qid= 0)
TX_cnt( 1): (pid= 0:qid= 0)
2, RX-TX
RX_cnt( 1): (pid= 1:qid= 0)
TX_cnt( 1): (pid= 1:qid= 0)

Port :
0, nb_lcores 1, private 0xaaaacefc92b0, lcores: 1
1, nb_lcores 1, private 0xaaaacefcf068, lcores: 2

Initialize Port 0 -- TxQ 1, RxQ 1
** Device Info (dpni.1, if_index:0, flags 00000066) **
min_rx_bufsize : 512 max_rx_pktlen :10240 hash_key_size : 0
max_rx_queues : 128 max_tx_queues : 16 max_vfs : 0
max_mac_addrs : 16 max_hash_mac_addrs: 0 max_vmdq_pools: 16
vmdq_queue_base: 0 vmdq_queue_num : 0 vmdq_pool_base: 0
nb_rx_queues : 0 nb_tx_queues : 0 speed_capa : 00005d60

flow_type_rss_offloads:000000029e03fffc reta_size : 0
rx_offload_capa :VLAN_STRIP IPV4_CKSUM UDP_CKSUM TCP_CKSUM OUTER_IPV4_CKSUM VLAN_FILTER JUMBO_FRAME SCATTER TIMESTAMP SCTP_CKSUM OUTER_UDP_CKSUM
tx_offload_capa :VLAN_INSERT IPV4_CKSUM UDP_CKSUM TCP_CKSUM SCTP_CKSUM OUTER_IPV4_CKSUM MT_LOCKFREE MULTI_SEGS MBUF_FAST_FREE
rx_queue_offload_capa :0000000000000000 tx_queue_offload_capa :0000000000000000
dev_capa :0000000000000000

RX Conf:
pthresh : 0 hthresh : 0 wthresh : 0
Free Thresh : 0 Drop Enable : 0 Deferred Start : 0
offloads :0000000000000000
TX Conf:
pthresh : 0 hthresh : 0 wthresh : 0
Free Thresh : 0 RS Thresh : 0 Deferred Start : 0
offloads :0000000000000000
Rx: descriptor Limits
nb_max :65535 nb_min : 0 nb_align : 1
nb_seg_max :65535 nb_mtu_seg_max :65535
Tx: descriptor Limits
nb_max :65535 nb_min : 0 nb_align : 1
nb_seg_max :65535 nb_mtu_seg_max :65535
Rx: Port Config
burst_size : 32 ring_size : 512 nb_queues : 1
Tx: Port Config
burst_size : 32 ring_size : 512 nb_queues : 1
Switch Info: (null)
domain_id :65535 port_id : 0

Create: Default RX 0:0 - Memory used (MBUFs 32768 x (size 1618 + Hdr 128)) + 192 = 55873 KB, headroom 128
Set RX queue stats mapping pid 0, q 0, lcore 1

Create: Special TX 0:0 - Memory used (MBUFs 256 x (size 1618 + Hdr 128)) + 192 = 437 KB, headroom 128

Port memory used = 56310 KB
Src MAC 00:00:00:c4:f9:1f
<Multicast mode Enabled>
Initialize Port 1 -- TxQ 1, RxQ 1
** Device Info (dpni.2, if_index:0, flags 00000066) **
min_rx_bufsize : 512 max_rx_pktlen :10240 hash_key_size : 0
max_rx_queues : 128 max_tx_queues : 16 max_vfs : 0
max_mac_addrs : 16 max_hash_mac_addrs: 0 max_vmdq_pools: 16
vmdq_queue_base: 0 vmdq_queue_num : 0 vmdq_pool_base: 0
nb_rx_queues : 0 nb_tx_queues : 0 speed_capa : 00005d60

flow_type_rss_offloads:000000029e03fffc reta_size : 0
rx_offload_capa :VLAN_STRIP IPV4_CKSUM UDP_CKSUM TCP_CKSUM OUTER_IPV4_CKSUM VLAN_FILTER JUMBO_FRAME SCATTER TIMESTAMP SCTP_CKSUM OUTER_UDP_CKSUM
tx_offload_capa :VLAN_INSERT IPV4_CKSUM UDP_CKSUM TCP_CKSUM SCTP_CKSUM OUTER_IPV4_CKSUM MT_LOCKFREE MULTI_SEGS MBUF_FAST_FREE
rx_queue_offload_capa :0000000000000000 tx_queue_offload_capa :0000000000000000
dev_capa :0000000000000000

RX Conf:
pthresh : 0 hthresh : 0 wthresh : 0
Free Thresh : 0 Drop Enable : 0 Deferred Start : 0
offloads :0000000000000000
TX Conf:
pthresh : 0 hthresh : 0 wthresh : 0
Free Thresh : 0 RS Thresh : 0 Deferred Start : 0
offloads :0000000000000000
Rx: descriptor Limits
nb_max :65535 nb_min : 0 nb_align : 1
nb_seg_max :65535 nb_mtu_seg_max :65535
Tx: descriptor Limits
nb_max :65535 nb_min : 0 nb_align : 1
nb_seg_max :65535 nb_mtu_seg_max :65535
Rx: Port Config
burst_size : 32 ring_size : 512 nb_queues : 1
Tx: Port Config
burst_size : 32 ring_size : 512 nb_queues : 1
Switch Info: (null)
domain_id :65535 port_id : 0

Create: Default RX 1:0 - Memory used (MBUFs 32768 x (size 1618 + Hdr 128)) + 192 = 55873 KB, headroom 128
Set RX queue stats mapping pid 1, q 0, lcore 2

Create: Special TX 1:0 - Memory used (MBUFs 256 x (size 1618 + Hdr 128)) + 192 = 437 KB, headroom 128

Port memory used = 56310 KB
Src MAC 00:00:00:c4:f9:20
<Multicast mode Enabled>
Total memory used = 112620 KB

Create: TX_MBUF - Memory used (MBUFs 32768 x (size 1618 + Hdr 128)) + 192 = 55873 KB, headroom 128
=== Display processing on lcore 0
RX/TX processing lcore: 1 rx: 1 tx: 1
For RX found 1 port(s) for lcore 1
For TX found 1 port(s) for lcore 1
RX/TX processing lcore: 2 rx: 1 tx: 1
For RX found 1 port(s) for lcore 2
For TX found 1 port(s) for lcore 2

 And the log of rte_eth_rx_burst. At dpdk/drivers/net/dpaa2/dpaa2_rxtx.c Line 369.
mXXXXX was printed before enqueue.
tXXXXX was printed after dequeue

t26436,17584dc00
t26437,17584e340
t26438,175918c00
PMD: to mbuf - mbuf =0x17584ea80, mbuf->buf_addr =0x17584eb00, off = 128,fd_off=128 fd =17584eb00, meta = 128 bpid =1, len=1370
m26439,17584ea80
PMD: to mbuf - mbuf =0x17592ae00, mbuf->buf_addr =0x17592ae80, off = 128,fd_off=128 fd =17592ae80, meta = 128 bpid =1, len=1370
m26440,17592ae00
PMD: to mbuf - mbuf =0x17584f1c0, mbuf->buf_addr =0x17584f240, off = 128,fd_off=128 fd =17584f240, meta = 128 bpid =1, len=1370
m26441,17584f1c0
PMD: to mbuf - mbuf =0x17584f900, mbuf->buf_addr =0x17584f980, off = 128,fd_off=128 fd =17584f980, meta = 128 bpid =1, len=1370
m26442,17584f900
PMD: to mbuf - mbuf =0x175850040, mbuf->buf_addr =0x1758500c0, off = 128,fd_off=128 fd =1758500c0, meta = 128 bpid =1, len=1370
m26443,175850040
PMD: to mbuf - mbuf =0x175850780, mbuf->buf_addr =0x175850800, off = 128,fd_off=128 fd =175850800, meta = 128 bpid =1, len=1370
m26444,175850780
PMD: to mbuf - mbuf =0x175850ec0, mbuf->buf_addr =0x175850f40, off = 128,fd_off=128 fd =175850f40, meta = 128 bpid =1, len=1370
m26445,175850ec0
PMD: to mbuf - mbuf =0x175851600, mbuf->buf_addr =0x175851680, off = 128,fd_off=128 fd =175851680, meta = 128 bpid =1, len=1370
m26446,175851600
PMD: to mbuf - mbuf =0x1758409c0, mbuf->buf_addr =0x175840a40, off = 128,fd_off=128 fd =175840a40, meta = 128 bpid =1, len=1370
m26447,1758409c0
PMD: to mbuf - mbuf =0x1758409c0, mbuf->buf_addr =0x175840a40, off = 128,fd_off=128 fd =175840a40, meta = 128 bpid =1, len=1370
m26448,1758409c0
PMD: to mbuf - mbuf =0x175821cc0, mbuf->buf_addr =0x175821d40, off = 128,fd_off=128 fd =175821d40, meta = 128 bpid =1, len=1370
m26449,175821cc0
PMD: to mbuf - mbuf =0x17590cf80, mbuf->buf_addr =0x17590d000, off = 128,fd_off=128 fd =17590d000, meta = 128 bpid =1, len=1370
m26450,17590cf80
PMD: to mbuf - mbuf =0x17590d6c0, mbuf->buf_addr =0x17590d740, off = 128,fd_off=128 fd =17590d740, meta = 128 bpid =1, len=1370
m26451,17590d6c0
PMD: to mbuf - mbuf =0x175814340, mbuf->buf_addr =0x1758143c0, off = 128,fd_off=128 fd =1758143c0, meta = 128 bpid =1, len=1370
m26452,175814340
PMD: to mbuf - mbuf =0x175912dc0, mbuf->buf_addr =0x175912e40, off = 128,fd_off=128 fd =175912e40, meta = 128 bpid =1, len=1370
m26453,175912dc0
t26439,17584ea80
t26440,17592ae00
t26441,17584f1c0
t26442,17584f900
t26443,175850040
t26444,175850780
t26445,175850ec0
t26446,175851600
t26448,1758409c0
=====seq num error exp 26447 but 26448
t26448,1758409c0
=====seq num error exp 26449 but 26448
t26449,175821cc0
t26450,17590cf80
t26451,17590d6c0
t26452,175814340
t26453,175912dc0
PMD: to mbuf - mbuf =0x1758290c0, mbuf->buf_addr =0x175829140, off = 128,fd_off=128 fd =175829140, meta = 128 bpid =1, len=1370
m26454,1758290c0
PMD: to mbuf - mbuf =0x17590e540, mbuf->buf_addr =0x17590e5c0, off = 128,fd_off=128 fd =17590e5c0, meta = 128 bpid =1, len=1370
m26455,17590e540

[yipingwang]

Which example/application are you running for this scenario? Or you wrote code by themselves? If yes, can you share your patch/code for us to review?