Configure Shared Peripheral Interrupt as Secure to Handle in S-EL1 or EL3

[matthias_staube]

Hello,

I'm trying to figure out how to configure and handle secure interrupts on i.MX 8. Here is what I have found out so far:

• GICv3 is initialized in EL3 by imx-atf through the following chain:

bl31_platform_setup
  --> plat_gic_driver_init
    --> gicv3_driver_init(&arm_gic_data);
  --> plat_gic_init
    --> gicv3_distif_init();                    // distributor interface
      --> gicv3_secure_spis_config_props        // Shared Peripheral Interrupts
    --> gicv3_rdistif_init(plat_my_core_pos()); // redistributor interface
    --> gicv3_cpuif_enable(plat_my_core_pos()); // CPU interface‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This arm_gic_data is initialized as follows:

static const interrupt_prop_t g01s_interrupt_props[] = {
 INTR_PROP_DESC(6, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP1S, GIC_INTR_CFG_LEVEL),
 INTR_PROP_DESC(7, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP0, GIC_INTR_CFG_LEVEL),
 INTR_PROP_DESC(78, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP1S, GIC_INTR_CFG_LEVEL),  // <-- my entry here
};

const gicv3_driver_data_t arm_gic_data = {
 .gicd_base = PLAT_GICD_BASE,
 .gicr_base = PLAT_GICR_BASE,
 .interrupt_props = g01s_interrupt_props,  // my entry is passed through this field
 .interrupt_props_num = ARRAY_SIZE(g01s_interrupt_props),
 .rdistif_num = PLATFORM_CORE_COUNT,
 .rdistif_base_addrs = rdistif_base_addrs,
 .mpidr_to_core_pos = plat_imx_mpidr_to_core_pos,
};‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

My assumption was that by adding this entry for the wdog0 interrupt (78), I can have it routed to EL3 which would eventually route it to the S-EL1 payload (OP-TEE). However, if I enable the watchdog0 the interrupt is routed to GNU/Linux, as is shown here:

# cat /proc/interrupts | grep wdog
 59:          0          0          0          0  GPC-PSCI  78 Edge      30280000.wdog
# [  194.865972] watchdog0: pretimeout event
# cat /proc/interrupts | grep wdog
 59:          1          0          0          0  GPC-PSCI  78 Edge      30280000.wdog‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Does anyone have experience with using secure interrupts and has an idea what I am still missing? I would expect the SoC to reset without the interrupt ever registering in GNU/Linux.

The wdog0 is just a more or less arbitrary example I chose because it would be triggered periodically.

Thank you in advance!

Edit: OP-TEE OS seems to do separate configuration of the GIC, and after inspecting CAAM driver code, I realized that 32 should be added to the actual interrupt number in OP-TEE, so 78 + 32 == 110. I do not understand why this is the case, but it likely has to do with the way OP-TEE accesses the GIC registers (32 is one register width).

Calling itr_add() in OP-TEE seems to apply the correct configuration, and the interrupt is no longer received inside GNU/Linux (the configuration in BL31 was most likely correct, too, but OP-TEE maybe just overrides everything as non-secure by default again).

However, now the interrupt is not passed to GNU/Linux, but I haven't figured out where it actually goes, since my OP-TEE registered handler is not being called, although itr_enable() is called, too.

[matthias_staube]

As mentioned in the edit to my original question, the offset 32 had to be added to the required interrupt ID from the reference manual, like so (both for BL31 and OP-TEE OS):

INTR_PROP_DESC(78 + 32, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP1S, GIC_INTR_CFG_LEVEL)‍

The method to configure the GIC via entries of the type above was correct, however, OP-TEE OS overrides the configuration of every interrupt to be non-secure, disabled, etc. Therefore, the configuration done in BL31 was overwritten.

My solution to handle the wdog interrupt inside the secure world was to configure it as Secure Group 0 instead, and register an appropriate interrupt handler in BL31. This way it was possible to periodically service the wdog interrupt inside the TrustZone secure world. I simply commented out the loop inside OP-TEE OS which overrides all default configurations inside the optee gic driver.

[devendradevadig]

Dear Sir,

As per your suggestion I tried to configure the interrupt 64 as in Secure Group 0. In i.MX8MQ I am trying for GPIO1_6 and IRQ number is 64.

static const interrupt_prop_t g01s_interrupt_props[] = {
INTR_PROP_DESC(6, GIC_HIGHEST_SEC_PRIORITY,
INTR_GROUP1S, GIC_INTR_CFG_LEVEL),
INTR_PROP_DESC(7, GIC_HIGHEST_SEC_PRIORITY,
INTR_GROUP0, GIC_INTR_CFG_LEVEL),
INTR_PROP_DESC( (64 + 32), GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP0, GIC_INTR_CFG_LEVEL), // <-- my entry here

};

Interrupt registration implementation added by refering below source code:

https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/nxp/common/setup/ls_interrupt_...

$cp "plat/nxp/common/setup/ls_interrupt_mgmt.c" "plat/imx/common/ls_interrupt_mgmt.c"

$cp "plat/nxp/common/setup/include/ls_interrupt_mgmt.h" "./plat/imx/common/include/ls_interrupt_mgmt.h"

Just after the GICv3 driver initialization, implemented the test interrupt handler for IRQ (64+96).

#ifdef IW_TEST_INTERRUPT
static uint64_t int64_interrupt_handler(uint32_t id, uint32_t flags,
void *handle, void *cookie)
{
ERROR("int64_interrupt_handler..!!!! id = %d \n", id);
return 0;
}
#endif

void bl31_platform_setup(void)
{
generic_delay_timer_init();

/* init the GICv3 cpu and distributor interface */
plat_gic_driver_init();
plat_gic_init();

/* gpc init */
imx_gpc_init();

dram_info_init(SAVED_DRAM_TIMING_BASE);

#ifdef IW_TEST_INTERRUPT
ERROR(" ls_el3_interrupt_config\n");
ls_el3_interrupt_config();
request_intr_type_el3((64 + 32), int64_interrupt_handler);
#endif

}

I commented the GIC configuration in OP-TEE.

Once booted with above changes, just after Linux OP-TEE Driver initialization I am getting intterupt prints continuously as below:

[ 2.902233] caam-snvs 30370000.caam-snvs: violation handlers armed - init state
[ 2.910650] hidraw: raw HID events driver (C) Jiri Kosina
[ 2.926934] usbcore: registered new interface driver usbhid
[ 2.932603] usbhid: USB HID core driver
[ 2.942859] usb 1-1: New USB device found, idVendor=0424, idProduct=2514
[ 2.949965] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 2.976422] ashmem: initialized
INFO: Interrupt recvd is 96
ERROR: int64_interrupt_handler..!!!! id = 96
INFO: Interrupt recvd is 96
ERROR: int64_interrupt_handler..!!!! id = 96
INFO: Interrupt recvd is 96
ERROR: int64_interrupt_handler..!!!! id = 96
INFO: Interrupt recvd is 96

...................

What is the reason I am getting interrupt continuosly here ?

Interrupt configuration is done as below:

set_interrupt_rm_flag(flags, NON_SECURE);
rc = register_interrupt_type_handler(INTR_TYPE_EL3,
ls_el3_interrupt_handler, flags);

Below is the code of interrupt handler:

static uint64_t ls_el3_interrupt_handler(uint32_t id, uint32_t flags,
void *handle, void *cookie)
{
uint32_t intr_id;
interrupt_type_handler_t handler;

intr_id = plat_ic_get_pending_interrupt_id();

INFO("Interrupt recvd is %d\n", intr_id);

handler = type_el3_interrupt_table[intr_id];
if (handler != NULL) {
handler(intr_id, flags, handle, cookie);
}/*
* Mark this interrupt as complete to avoid a interrupt storm.
*/
plat_ic_end_of_interrupt(intr_id);

return 0U;
}

I had attached the log for reference. Please can you suggest me what changes required for getting interrupt properly ?

Thanks and Regards,

Devendra Devadiga

[gusarambula]

Hello Matthias Stauber, 

Thank you for posting your findings. I'm sure they will help other Community Users!

Regards.