iMX6: MSI-X not working

[concerned12345]

I am using an iMX6 quad core running Linux v4.1.15 (Yocto) as a root complex. Currently whenever I try to get MSI-X implemented it fails even though the same driver code works when running my hardware with an Intel i7 (kernel v3). The APIs that fail are:

• pci_enable_msix()
• pci_enable_msix_range()
• pci_enable_msix_exact()

All of the above fail and return EINVAL (22). I am able to get a single MSI (non -X) working on the iMX6 with pci_enable_msi(). When running my driver on the iMX6 the pci_msix_vec_count() returns the correct MSI-X count which can also be verified through "lspci -v". According to the documentation MSI-X should be functional (as well as multi MSI).

Questions:

1. Does the iMX6 support MSI-X?
2. Does the iMX6 support multi MSI (non -X)?
3. Are there any kernel flags that need to be set in the configuration?

Update:

I started looking through the kernel code to see where a EINVAL could be returned. Now, I am not a Linux expert so the following assumption may be wrong. When going down the path:

• pcie_enable_msix_range()
  • msix_capability_init()
    • pci_msi_setup_msi_irqs()
      • arch_setup_msi_irq()  <-- HERE

I noticed that for x86 and PowerPC the arch_setup_msi_irq() is overridden under /arch/x86/kernel/x86_init.c and /arch/powerpc/kernel/msi.c. Within the generic function listed in /drivers/pci/msi.c there is a comment under arch_setup_msi_irqs() that states:

If an architecture wants to support multiple MSI, it needs to override arch_setup_msi_irqs()

I did a search across the Freescale kernel source and I could not find an override for the above function. This leads me to believe that MSI-X and multi MSI (non -X) are not implemented for the iMX6.

[riyazurrahman]

Hi Fabio,

Thanks for providing this info. We are working with 4.9.11 kernel version. Is there any patch-set available for this version?

Best Regards,

Riyaz

[fabio_estevam]

Hi Riyaz,

This patch series is not availabe on 4.9. It has not even reached mainline yet, so you will need to use a mainline kernel and manually apply them.

[concerned12345]

Sorry for the late reply but I already solved this sometime back with patches. Please see my post:

SOLVED: i.MX6 and Multi MSI Working

[riyazurrahman]

I am also checking for support of MSI-X in iMX6.

Could you please let me know whether iMX6 has support for MSI-X interrupts? If yes, apart from enabling kernel configuration CONFIG_PCI_MSI=y, any other extra configurations required?

Best Regards,

Riyaz  

[fabio_estevam]

Hi Riyaz,

You can try a mainline kernel with apply this series:

[PATCH v8 0/9] PCI: dwc: Enables MSI-X driver support — Linux PCI 

[igorpadykov]

Hi concerned12345

according to i.MX6SX Reference Manual it is supported only in EP mode.

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[concerned12345]

Hi Igor,

In the line above the one you've highlighted it states:

MSI interface is used only in EP mode

I have gotten single MSI to work by calling pci_enable_msi() while the iMX6 is acting as a root complex. If the iMX6 can receive MSI messages from an EP device as RC it should be able to receive MSI-X and multi MSI. I noticed that document is for the solo and I'm running the quad although I doubt there is much difference in this department.

If the document assumes MSI only in EP when I've gotten it to work in RC could it also be missing RC information about MSI-X?

When looking at revision 4 (p 4190) of the same document I found MSI-X in RC for the iMX6Q:

I also found the following on page 4117 of the rev. 4 document which states the iMX6 in RC mode can support/receive 32 interrupts per EP:

Thank you

[igorpadykov]

Hi concerned12345

Table 48-2 rev.4 states the same, i.MX6Solo PCIe is the same as i.MX6Q.

page 4117 gives generic IP memory map, particular i.MX6 implementation may lack some features,

as stated on Table 48-2.

Best regards
igor

[concerned12345]

Hi Igor,

When looking at table 48-2 of rev. 4 on page 4053 I see it also says:

MSI interface is used only in EP mode

In my previous post I stated that I am able to get MSI working when in RC mode so that above statement is false. Was table 48-2 of rev. 4 copied over from table 46-2 of rev. 1 without taking into account updates? I know there have been fixes to the PCIe driver code since kernel v3 for the iMX6.

I don't want this thread to turn into us quoting the spec sheet, is there a way to contact an engineer and ask if MSI-X and multi MSI (non -X) is supported an if so has it been tested with real hardware?

Thank you.