I decided to switch from the linux-rel_imx-3.14.52_ga kernel to the linux-rel_imx-4.1.15_ga kernel for use with the imx6sx sabre board. The 4.1.15 kernel built with no problems and the board boots up properly and runs as expected. However when I tried suspend and resume I ran into problems resuming. Note the suspend to mem (echo mem > /sys/power/state) also did not drop the current usage as low as it did in 3.14.52 either.
The problems looked similar to the problems I was having on the 3.14.52 kernel resuming (kernel would hang, the mmc card would have issues, or whenever I would access my encrypted partition errors would occur and need to reboot) before I did the following as documented in the i.MX Linux Reference Manual:
The full CAAM function is exclusive with the Mega/Fast mix off feature in DSM. If
CAAM is enabled, the Mega/Fast mix off feature needs to be disabled, and the user
should "echo enabled > /sys/bus/platform/devices/2100000.aips-bus/2100000.caam/
2101000.jr0/power/wakeup" after the kernel boots up, and then Mega/Fast mix will
keep the power on in DSM.
With the 3.14.52 kernel everything was fine after I enabled the wakeup as suggested above.
I tried doing the same thing with the 4.1.15 kernel, but it has no impact any more. After some digging I found the following patch which appears to have changed this functionality by Marc Zyngier titled: ARM: imx6: convert GPC to stacked domains. It modifies the device tree interrupt structure to use stacked domains (not sure I totally understand this). Given these modifications I was wondering if anyone has a new workaround for this?
I did debug this and found that in arch/arm/mach-imx/gpc.c function imx_gpc_mf_mix_off() it no longer returns before turning off mega/fast mix in 4.1.15 like it did in 3.14.52 when the wake was enabled. I did hard code it to return before turning off mega/fast mix in 4.1.15 and that appears to fix the issue (current usage in suspend was lower and resumed properly, but not a real solution). Anyway I am going to keep working at this, but again I was wondering if anyone else had already encountered and fixed this problem. It is really noticeable if there is an encrypted file system partition and you attempt to access it after resuming. Also if you do the old 3.14.52 workaround, the resume code in 4.1.15 complains/warns about an "Unbalanced IRQ wake disable".
Also note that when you dump /proc/interrupts in 4.1.15 that the only GIC interrupts are now IRQ 309 2101000.jr0, IRQ 310 2102000.jr1, and IRQ 311 PCIe PME (almost all the rest are now GPC). In 3.14.52 almost all the interrupts were GIC (including those listed above, IRQ 137 2101000.jr0, IRQ 138 2102000.jr1, and IRQ 155 PCIe PME) and none were GPC. Looking at the IRQ masking registers 1-4 it looks like interrupts with IRQ numbers above 159 are not represented (maybe that is why the workaround no longer works because the IRQ is 309).