NXP LS1012a FRDM - GIC Virtualization

[dloosli]

Hi everyone

I am still trying to port an OpenSource hypervisor (i.e. Muen SK) to the ARMv8-A AArch64 architecture and decided to use the NXP LS1012a FRDM evaluation board as the target platform. The Kernel is written in Ada/SPARK.

Everything went well so far and I am now working on the implementation of the generic interrupt controller (GIC). While the physical interrupts are working perfectly, I have some problems with the maintenance interrupts for the virtual CPU interface (official ARM PPI Interrupt ID 25). As soon as a maintenance interrupt occures, the code execution switches correctly from the VM code at Exception Level 1 to the hypervisor code at EL2. When I acknowledge the maintenance interrupt, it also changes its state correctly from pending to active. But it somehow reasserts immediately again to the state "pending and active". I then tried to clear the pending state by writing to the according ICPENDING register bit of the distributor, but still no success. I now found in the QorlQ LS1012A Reference Manual, chapter 5.2 (interrupt assignment), an interrupt entry "Virtual CPU interface maintenance interrupt core 0" with ID 229 - but this one does never switch to pending state.

So my questions - what is this SPI ID 229 interrupt for? And how do I have to use it for maintenance interrupt handling? Or can I manually "deassert" the official maintenance interrupt with ID 25?

I would appreciate your advices!

Thanks, David

[dloosli]

Hi everyone

Even though I still can't explain the phenomenon completely and don't know the purpose of the NXP SPI 229, I found two ways to solve the problem.

First, I have to say that for the original hypervisor implementation I used the GIC EOI mode 0 - i.e. writing the interrupt ID of the pending interrupt to the EOIR register sets the interrupt state directly to inactive. In contrast, the ARM GICv2 architectur specification (@ufedor the GIC-400 maintenance interrupt is officially also mapped to PPI ID 25) recommends for a hypervisor implementation to use GIC EOI mode 1 - i.e. writing the interrupt ID of the pending interrupt to the EOIR only leads to a priority drop of the according interrupt and one has to deactivate it after the VM has handled the virtual interrupt by writing the interrupt ID to the GICC DIR register (c.f. ARM GICv2 architecture specification, chapter 5).

After some tests, I finally found out that by using the GIC EOI mode 1 one can handle the maintenance interrupt on the NXP LS1012a FRDM board as described in the ARM specification documents without any "reasserting problems", in short: (1) Acknowledge the pending interrupt; (2) Inject the virtual interrupt into the VM by (a) setting the accoring virtual IRQ bit in the HCR_EL2 register, (b) enabling the maintenance interrupts in the GIC virtual control register and (c) write the virtual IRQ vector to the GIC LR list register; (3) Drop priority by writing to the GICC EOIR register and hand over execution to the VM; (3) as soon as the VM has handled the interrupt (i.e. written to vEOI) a maintenance interrupt is triggered and execution returns to the hypervisor; (4) Disable the physical interrupt by writing to the GICC DIR register and reset the virtual registers (c.f. step 2).

If someone wants to use the GIC EOI mode 0 (even though ARM doesn't recommend it) for the hypervisor implementation but faces the same "reasserting problem" with the maintenance interrupt as I experienced, I can provide a workaround. As soon as the execution returns to the hypervisor due to the triggered maintenance interrupt, one can (1) completely disable the maintenance interrupt ID 25 (i.e. physically on the distributor); (2) do the steps 3 to 4 as described above; and (3) finally reenable the maintenance interrupt ID 25. As I said, I can neither explain the "reasserting problem" itself nor the workaround for GIC EOI mode 0 - but I am gonna keep an eye on it and will (if I can find some time) test it on other evaluation boards and setups because my development environment uses JTAG debugging over OpenOCD and a customized U-Boot bootloader.

Thanks again, David

[dloosli]

Hi Ufedor

Thanks for your answer. But I am quite sure that this isn't an ARM related issue. The ARM GIC-400 reference manual only specifies the first 32 interrupts. All other interrupts are connected to peripheral devices and specified by the manufacturer (c.f. QorlQ LS1012A Reference Manual, chapter 5.2, with reference to the ARM GIC-400 RM for the first 32 interrupts). Therefore I think that the interrupt "Virtual CPU interface maintenance interrupt core 0" with ID 229 should be NXP LS1012a related - but I couldn't find any hint in the QorlQ LS1012A Reference Manual.

Thanks again, David

[ufedor]

In GIC-500, this is PPI and mapped on “25”, but in LS1012A, GIC-400 does not have this as native PPI, so mapped on the SPI 229.

Please consult with ARM concerning the Virtual CPU interface maintenance interrupt handling.

[ufedor]

Please consider discussing the issue with ARM technical support directly.