3 FMAN portals for 4 cores in LSDK DPDK userspace?

[mark_callaghan]

Can I specifically allocate 3 FMAN portals between 4 cores in DPDK userspace, where one core does not perform io?

I am planning a product based on LS1046A, using Ubuntu and DPDK. I would like to use 3 cores in DPDK userspace for actual frame handling, and one core for management and reporting with a thread in the same memory space as the frame handling cores. My application uses 2 ethernet ports for through traffic.

I also need an ethernet port for the Linux system (running on core 0), so I have modified the dtb and usdpaa config xml files to allocate one port for linux, 3 for DPDK userspace (as in LSDKUG_2004 sect 9.2.4.3).

My initial thought was that core 0 would be used by Linux and the DPDK management thread, and I would issue the kernel parameter isolcpus=1-3, and start the DPDK app giving it 4 cores with coremask -c 0xf.

However, I think the LS1046A has only 4 FMAN portals? So Linux will consume one portal, leaving three for DPDK userspace?

Can I determine which cores have access to the FMAN portals in DPDK userspace? So that Core 0 does not use an FMAN portal in userspace, leaving three for the frame handling cores?

Or will core 0 inherit the same FMAN portal in userspace as it has under the linux kernel?

I'd be grateful for any pointers/advice.

Cheers,

Mark

[yipingwang]

FMAN portals are in fact qman/bman portals. On LS1046A there are a total of 10 qbman sw portals.

When an application -i.e testpmd, is run in dpdk a parameter - called lcore can be given as argument.

According to DPDk: "lcore refers to a logical execution unit of the processor, sometimes called a hardware thread."

For each core in the system you can assign at least one thread.

(see https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html)

Example ./testpmd  --lcores='1,2,3,4@1'  \ --master-lcore 0

In the above example lcore 4 which is thread 4 is spawned on core 1.

Thread 0 is  spawned on core 0(master-lcore) while 1,2 and 3 are spawned on cores 1,2 and 3.

Each thread: 1,2,3 and 4 are consumer threads that can dequeue from queues affine to cores 1,2 and 3.

In the dpaa inplementation ....for a consumer thread, affine to a cpu that consumes from a queue this translates in calling rte_dpaa_portal_init(for example in dpaa_rxtx.c which deals with the frames'reception) which will try to allocate a qman / bman portal.

In your case you will have 3 cores used exclusively by dpdk (no scheduling is performed on them due to isolcpus) while core 0 is under Linux scheduler control

To summarie:

-in your implementation you can use the testpmd model where the argument lcore is given(if lcore is used you can no longer use -c) with the lcore ids for consumer threads while master-lcore is the control thread.

[mark_callaghan]

Awesome. Thanks for your help.