DPAA QMAN Order Restoration

[sedat_altun]

"DPAA QMAN Order restoration implementation in SDK".

We are using DPAA drivers on t4240 SoC with sdk 2.0 for a network device like a router.

Our problem is the difference between ingress and egress packet order. They are not the same . May be this is natural due to multicore architecture of T4240. But our TCP applications are sensitive to out of order packets. As a result we get low performance like 700Mbps on a 10g port.

NXP application notes suggest us to define flows and affine each of them to a single core for suppressing order problems within a single flow.  But in this solution the performance of one flow is limited by the single core. But our performance requirements are not met with using a single core for a single flow. We want all the cpu cores to handle the incoming traffic in parallel to get higher data rates. 

Is there any other solution for the packet misordering problem than assigning a flow to a single core.

In DPAA Reference manuel  there is a solution  called Qman order restoration which is used for ordering the out of ordered packtes before sending to FMAN ports.

Can we use this method for packet ordering ?

If yes how can we enable this restoration method in SDK?

Best regards.

[yipingwang · Accepted solution]

Hello Sedat Altun ,

The FMAN PCD can be configured to either directly match a flow to a core or to use the hashing to provide traffic spreading that offer a permanent flow-to-core affinity through dedicated channel.

If you need the pool channel approach where multiple cores may pool together to service a specific set of flows.

Order is preserved as long as two or more cores never process frames from the same flow at the same time. This can also be accomplished by using hold active scheduling along with discrete consumption acknowledgment (DCA) mode associated with the DQRR. Although flow affinity may change for an FQ with hold active scheduling when the FQ is emptied, if the new work (from frames received after the FQ is emptied) is held off until all previous work completes, then the flow will not be processed by multiple cores simultaneously, thereby preserving order.

You could enable DCA mode when invoke qman_enqueue with DCA related flags specified  in Linux Kernel driver.

#define QMAN_ENQUEUE_FLAG_DCA 0x00008000 /* perform enqueue-DCA */
#define QMAN_ENQUEUE_FLAG_DCA_PARK 0x00004000 /* If DCA, requests park */
#define QMAN_ENQUEUE_FLAG_DCA_PTR(p) /* If DCA, p is DQRR entry */ \
(((u32)(p) << 2) & 0x00000f00)

int qman_enqueue(struct qman_fq *fq, const struct qm_fd *fd, u32 flags);

Thanks,

Yiping

[sedat_altun]

Hello,

Thank you for your help.

I have 2 questions related to your suggestions.

1- If we use pool channel method instead of dedicated cahnnel with pcd frames and activate DCA can we maintaion also the  engress flow order or it is just for ingress packet order within a flow?

2-  Which below flag do I have to pass as a parameter to the qman_enqueu command

 QMAN_ENQUEUE_FLAG_DCA

or
 QMAN_ENQUEUE_FLAG_DCA_PARK 

or
 QMAN_ENQUEUE_FLAG_DCA_PTR(p)

Thank you very much.