i.MX8M Mini PCI Interrupts

[roneisworth]

We are trying to achieve WiFi6e and WiFi7 speeds over PCIe on a i.MX8M Mini based single board computer. So far we have been able to achieve 800 megabits per second, but typically should be able to reach higher, even up to 2800 mbps. We are wondering if the limitation is via a interrupt bottleneck. We were hoping to distribute the interrupts across multiple cores to achieve a higher throughput, but tools such as smp_affinity don't seem to provide that option. Is it true that 'smp affinity' works only for interrupts being handled by the ARM GIC (Interrupt Controller) which is why this cannot be changed for PCI interrupts? 

Is this just a limitation of NXP's PCI controller and implementation? Or are we missing any kernel configuration here?

See below output of interrupts. 

root@jammy-ubuntu:~# cat /proc/interrupts
            CPU0       CPU1       CPU2       CPU3
228:          4          0          0          0   PCI-MSI 1572864 Edge      bhi
229:          5          0          0          0   PCI-MSI 1572865 Edge      mhi
230:         39          0          0          0   PCI-MSI 1572866 Edge      mhi
231:         77          0          0          0   PCI-MSI 1572867 Edge      ce0, ce7
232:         77          0          0          0   PCI-MSI 1572868 Edge      ce1, ce14
233:       6217          0          0          0   PCI-MSI 1572869 Edge      ce2
234:          1          0          0          0   PCI-MSI 1572870 Edge      ce3
235:          2          0          0          0   PCI-MSI 1572871 Edge      ce5
236:       4896          0          0          0   PCI-MSI 1572872 Edge      DP_EXT_IRQ
237:          2          0          0          0   PCI-MSI 1572873 Edge      DP_EXT_IRQ
238:        586          0          0          0   PCI-MSI 1572874 Edge      DP_EXT_IRQ
239:     488757          0          0          0   PCI-MSI 1572875 Edge      DP_EXT_IRQ
240:       1333          0          0          0   PCI-MSI 1572876 Edge      DP_EXT_IRQ
241:       6688          0          0          0   PCI-MSI 1572877 Edge      DP_EXT_IRQ
242:       1333          0          0          0   PCI-MSI 1572878 Edge      DP_EXT_IRQ
243:       1333          0          0          0   PCI-MSI 1572879 Edge      DP_EXT_IRQ

[JorgeCas]

Hello,

You have reached the maximum bandwidth of PCIe interface. According to our results, the maximum bandwidth is around 768 - 768 MB/s for read and 809 - 441 MB/s for write for i.MX8M devices.

Best regards.

[roneisworth]

Jorge,
Are the number you provided for the i.MX8M Plus with PCIe 3.0?

Do you also have numbers for the i.MX8M Mini with PCIe 2.0?

Thanks!