Can I specifically allocate 3 FMAN portals between 4 cores in DPDK userspace, where one core does not perform io?
I am planning a product based on LS1046A, using Ubuntu and DPDK. I would like to use 3 cores in DPDK userspace for actual frame handling, and one core for management and reporting with a thread in the same memory space as the frame handling cores. My application uses 2 ethernet ports for through traffic.
I also need an ethernet port for the Linux system (running on core 0), so I have modified the dtb and usdpaa config xml files to allocate one port for linux, 3 for DPDK userspace (as in LSDKUG_2004 sect 9.2.4.3).
My initial thought was that core 0 would be used by Linux and the DPDK management thread, and I would issue the kernel parameter isolcpus=1-3, and start the DPDK app giving it 4 cores with coremask -c 0xf.
However, I think the LS1046A has only 4 FMAN portals? So Linux will consume one portal, leaving three for DPDK userspace?
Can I determine which cores have access to the FMAN portals in DPDK userspace? So that Core 0 does not use an FMAN portal in userspace, leaving three for the frame handling cores?
Or will core 0 inherit the same FMAN portal in userspace as it has under the linux kernel?
I'd be grateful for any pointers/advice.
Cheers,
Mark
FMAN portals are in fact qman/bman portals. On LS1046A there are a total of 10 qbman sw portals.
When an application -i.e testpmd, is run in dpdk a parameter - called lcore can be given as argument.
According to DPDk: "lcore refers to a logical execution unit of the processor, sometimes called a hardware thread."
For each core in the system you can assign at least one thread.
(see https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html)
Example ./testpmd --lcores='1,2,3,4@1' \ --master-lcore 0
In the above example lcore 4 which is thread 4 is spawned on core 1.
Thread 0 is spawned on core 0(master-lcore) while 1,2 and 3 are spawned on cores 1,2 and 3.
Each thread: 1,2,3 and 4 are consumer threads that can dequeue from queues affine to cores 1,2 and 3.
In the dpaa inplementation ....for a consumer thread, affine to a cpu that consumes from a queue this translates in calling rte_dpaa_portal_init(for example in dpaa_rxtx.c which deals with the frames'reception) which will try to allocate a qman / bman portal.
In your case you will have 3 cores used exclusively by dpdk (no scheduling is performed on them due to isolcpus) while core 0 is under Linux scheduler control
To summarie:
-in your implementation you can use the testpmd model where the argument lcore is given(if lcore is used you can no longer use -c) with the lcore ids for consumer threads while master-lcore is the control thread.
Awesome. Thanks for your help.