AnsweredAssumed Answered

USDPAA order restoration does not work

Question asked by ömer faruk er on Oct 9, 2019
Latest reply on Oct 17, 2019 by Yiping Wang

Hi,

I have a TCP/UDP performance problem while using T4240RDB and SDK 2.0 linux kernel. I am using the RDB board as a router, basicly the board is routing the traffic  received from eth0 to eth1.

There is one iperf client and one server on windows Host connecting over T4240RDB.

The performance of iperf is nearly 700Mbps.

Then I found that the main reason behind this low performance value is the packet ordering. Since T4240 is a multicore cpu the ingress packet order and the egress packet order is not same.

The first solution that we found is using fmc to affine tcp/udp flows to a single core for order preservation.

In this solution the performance for a single  tcp flow could not reach even 2 gbps which is not enough for us.

 

After some research we found that the order restoration mechanism can be used for sorting reordered packets.We asked NXP how to enable this mechanism in Linux kernel, they told us it is not supported in linux kernel instead you have to use USDPAA for this mechanism.

 

We started to use USDPAA ipsecfwd_app application in order to understand how order preservation and restoration mechanisms are work We  measured the TCP performance with and without order preservation/restoration.
The results was not as expected.  The TCP performance for ipsecfwd_app is 2.4Gbps with or without using order restoration mechanism. Also we are seeing out of order packets even enabling  order restoration mechanism in USDPAA.
Below you can find what we have done for enabling order preservation/restoration in usdpaa. If there is an error we will be appreciated if you tell us.
//ORDER RESTORATİON
1 - I've changed ppac.h file for order restoration as stated  in SDK document

 

   #undef PPAC_ORDER_RESTORATION
   to;
   #define PPAC_ORDER_RESTORATION

 

2 -  I built  usdpaa apps. I use SDK2.0 on T4240RDB.

 

I got 2.4Gbps TCP performance , and there are out of order packets.

//ORDER PRESERVATİON

 

I also build the usdpaa-apps for PPAC_ORDER_PRESERVATION. In SDK document it says:

1 - change ppac.h as stated in SDK document.

#undef PPAC_HOLDACTIVE /* Process each FQ on one portal at a time */
#undef PPAC_ORDER_PRESERVATION /* HOLDACTIVE + enqueue-DCAs */
Change the above to
#define PPAC_HOLDACTIVE /* Process each FQ on one portal at a time */
#define PPAC_ORDER_PRESERVATION /* HOLDACTIVE + enqueue-DCAs */

2 -  I built  usdpaa apps. I use SDK2.0 on T4240RDB.

I got 3 Gbps TCP performance  and there is no out of order packets.

 

If I am doing right for enabling order restoration in USDPAA why I am seeing out of order packets and why the TCP performance of order restoration  is is lower than order preservation value even pool channels are using in order restoration mechanism.

 

Why ORDER_PRESERVATION way better than ORDER_RESTORATION in speed and packet ordering ?

 

Any help will be appreciated.

 

Thanks.

Outcomes