udp packets received are out-of-order on B4860

[xunxiao]

I have two B4860 boards, A and B, fm1-mac5 of A is shared mac device,fm1-mac5 of B is private mac device,and the two NICs are connected directly through fiber.

I send udp packets from A to B, and use tcpdump to capture packets on B, Some packages are out-of-order. I can confirm that packets of B are sent in order. I wonder what goes wrong, whether i miss some setting of the NICs

Thanks,
Kession

[yipingwang · Accepted solution]

Hello xun xiao,

Unlike TCP, UDP is an unreliable high level protocol, we also encountered out-of-order issue in iperf UDP testing on B4860.

You could use core-affined queue to implement order order preservation, please use one of the following default FMC policy files.
/etc/fmc/config/private/b4860qds/*/policy_ipv4.xml
/etc/fmc/config/private/b4860qds/*/policy_ipv6.xml

Please execute FMC commands as the following before all networking related operation.
# fmc -c /etc/fmc/config/private/b4860qds/*/config.xml -p /etc/fmc/config/private/b4860qds/*/policy_ipv4.xml -a

This is because in DPAA1, each interface used by default one pool channel across all software portals and also the dedicated channel of each CPU. In Linux Kernel, PCD Frame queues in use dedicated channels.You could refer to
the section "5. Dedicated and Pool Channels Usage in Linux Kernel" inUsing QMAN Dedicated and Pool Channels in USDPAA and Linux Kernel .

You need to use multiple flows, after executing FMC policy, one flow will bind to one core, so all 4 cores will be used by multiple flows. In the real scenario, one user application uses one flow.

Have a great day,
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------