Queue Allocation Limitations on LX2160A Custom Board with Multiple Ethernet Interfaces

[o_m_v]

Hardware:

• CPU: LX2160A
• Board: Custom board with 16 1000BaseX SFP Ethernet ports (DPMAC3-DPMAC18)
• SerDes1 Protocol: 4 (8x 1G Ethernet)
• SerDes2 Protocol: 9 (8x 1G Ethernet)
• SerDes3 Protocol: 3 (2x PCIe)

Software:

• Bootloaders and Linux firmware are based on "LSDK-21.08" branches
• MC Firmware Version: v10.39.0

We have attached a diagram illustrating our DPL configuration, as well as the DPL DTS file obtained using the command restool dprc generate-dpl dprc.1 for a setup with 12 queues per interface.

---

Issue Description:

We can successfully allocate 8 queues (1 traffic class) for each of the 16 interfaces (DPMAC3-DPMAC18) without issues. However, when attempting to allocate 12 queues (1 traffic class) per interface, we receive the following error messages in the MC log:

[E, allocate_resource:1114] resman_bind() failed for DCP-8WQ
[E, allocate_channel_resources:168, DPCON] Ignore above error message, continue with 8WQ channel allocation...

Despite these errors, all ports appear to work correctly.

Questions:

1. Is it safe to use this configuration with 12 queues per interface, or do the error messages indicate a potential problem?
2. If there is an issue, can you suggest a way to resolve it?

---

When attempting to allocate even more queues (14 or 16 per interface), we encounter "No resources" errors, resulting in the failure to create some interfaces:

root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.18

root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.17
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.16
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.15
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.14
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.13
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.12
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.11
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.10
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.9
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.8
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.7
root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.6

root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.5
MC error: No resources (status 0x8)
Error: dpcon object was not created!

root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.4
MC error: No resources (status 0x8)
Error: dpcon object was not created!

root@TinyLinux:~# ls-addni -nq=16 -t=1 dpmac.3
MC error: No resources (status 0x8)
Error: dpcon object was not created!

Further Questions:

1. What is the maximum number of queues we can allocate for all 16 interfaces?
2. Is there a specific limit on queue allocation, and is it possible to increase this limit to allocate more queues per interface?

[yipingwang · Accepted solution]

For this case, suggest modifying the DPC DTS as below,
- wq_ch_conversion = <32>;
+ wq_ch_conversion = <64>;

Refer to the following note from this section 8.3.2.3.3.1 DPAA2 objects dependencies in LSDKUG2108.pdf.
Note that DPIO objects themselves transparently consume DPCONs (one
per DPIO object), which therefore must be subtracted from the total
number available to the DPNIs (they need not be explicitly declared in
the DPL, but they are simply not available to the rest of the system).
The system can provide up to 64 8-WQ DPCONs (and up to 256
2-WQ DPCONs and combinations thereof).