Hi, Nathan
Which kernel are you using, I think you can try disable WAIT mode.
For L3.10, you can comment out BM_CLPCR_ARM_CLK_DIS_ON_LPM setting in arch/arm/mach-imx/pm-imx6.c->imx6_set_lpm->WAIT_UNCLOCKED;
For L3.0.35, you can make sure bit5 of CCM_CLPCR is clear, just clear it after kernel boot up, or just pass "enable_wait_mode=off" in kernel command line to disable WAIT mode.