This discussion is complementing my original "Implementing PCIe support for bridges in iMX6Q WEC7 BSPs" post.
There is one final hurdle to be passed when bridge is connected to iMX6 PCI port - and it is IRQ handling.
Endpoint devices may be enumerated and identified, and their drivers activated and properly loaded, yet they would not operate properly if IRQs are not handled properly.
By design, PCIe bus devices should be able to share interrupts, and there are four interrupt lines on it - INT_A#, INT_B#, INT_C#, INT_D#.
All bridges and devices share these lines and in addition to that, bridges "scramble" the original INT_X# assignments of their endpoints.
Each INT_X# corresponds to one and only one IRQ in the BSP; for PCI bus there are four unique IRQs used by all devices on the PCI bus.
The routing algorithm is described in "PCI-to-PCI Bridge Architecture Specification", chapter 9.1 - "Interrupt Routing".
It depends on the downstream port number (remember, downstream ports numbers are assigned by the bridge and may not be sequential) and the formula is this:
new INT_Y# = 1 + ( (downstream port number MOD 4) + (original INT_X# - 1) ) MOD 4;
The matter is slightly complicated because PCI endpoints use numbers from 0 to 4 to identify the interrupt line used, 0 is for no interrupts, 1-4 correspond to A-D.
The original PCI Bus Driver is not capable of handling this translation; it uses the original IRQ from the endpoint as resource, which may not correspond to the actual IRQ generated.
In addition to this, PCI Bus Driver would not allow IRQ resource sharing - so this should be explicitly enabled in it for PCI Bus IRQs so that SYSINTs can be assigned.
Remember that the Interrupt Line register in the PCI Configuration space for all PCI devices is R/O and cannot be re-assigned!
(This is true for x86 platforms as well, so PCI Bus Driver in the PUBLIC may be considered partially crippled.)
That old crippled PCI Bus Driver may still be used if the hardware design carefully places each device in such downstream port that no two ever share interrupt, and Registry explicitly overrides the IRQ value for the device. Yet, PCI Bus Driver may refuse to enable second PCI device requesting same IRQ.
For example, if there are four endpoints connected to downstream ports 1, 4, 5 and 10, and all these endpoints have INT_A# as their interrupt line, then actual IRQs would be:
INT_B#, INT_A#, INT_B# and INT_C#
Even after PCI bus Driver gets properly modified to handle Interrupt Routing for bridges, the endpoints will need implementation of their own ISR drivers.
The handler in the ISR DLL must
- identify if the device is indeed interrupting, and
- preserve the value of its interrupt status register.
If the device is not interrupting, then SYSINTR_CHAIN should be returned to allow other devices sharing same IRQ to get a chance.
If the device is indeed interrupting, then ISR handler must assert its own SYSINT and make sure that the device driver's DPC (Deferred Procedure Call) will have access to the actual interrupt status bits from the time interrupt has been recognized. This would not be a problem for devices which require explicit interrupt status write-to-clear, but is challenge for devices which clear-on-read interrupt bits.
Once this gets done, then the device driver infrastructure is fully in place and the implementer can focus on the plain good old MMIO read and write, so that the PCI endpoint device behind the bridge can be operational! Good luck!
Ilko Dossev
Senior Software Engineer
Adenoe Embedded US
