IMX6SX Kernel 4.1.15 suspend/resume issue

cancel
Showing results for 
Search instead for 
Did you mean: 

IMX6SX Kernel 4.1.15 suspend/resume issue

2,416 Views
daveebright
Contributor III

I decided to switch from the linux-rel_imx-3.14.52_ga kernel to the linux-rel_imx-4.1.15_ga kernel for use with the imx6sx sabre board.  The 4.1.15 kernel built with no problems and the board boots up properly and runs as expected.  However when I tried suspend and resume I ran into problems resuming.  Note the suspend to mem (echo mem > /sys/power/state) also did not drop the current usage as low as it did in 3.14.52 either.

The problems looked similar to the problems I was having on the 3.14.52 kernel resuming (kernel would hang, the mmc card would have issues, or whenever I would access my encrypted partition errors would occur and need to reboot) before I did the following as documented in the i.MX Linux Reference Manual:

The full CAAM function is exclusive with the Mega/Fast mix off feature in DSM. If

CAAM is enabled, the Mega/Fast mix off feature needs to be disabled, and the user

should "echo enabled > /sys/bus/platform/devices/2100000.aips-bus/2100000.caam/

2101000.jr0/power/wakeup" after the kernel boots up, and then Mega/Fast mix will

keep the power on in DSM.

With the 3.14.52 kernel everything was fine after I enabled the wakeup as suggested above.

I tried doing the same thing with the 4.1.15 kernel, but it has no impact any more.  After some digging I found the following patch which appears to have changed this functionality by Marc Zyngier titled: ARM: imx6: convert GPC to stacked domains.  It modifies the device tree interrupt structure to use stacked domains (not sure I totally understand this).  Given these modifications I was wondering if anyone has a new workaround for this?

I did debug this and found that in arch/arm/mach-imx/gpc.c function imx_gpc_mf_mix_off() it no longer returns before turning off mega/fast mix in 4.1.15 like it did in 3.14.52 when the wake was enabled.  I did hard code it to return before turning off mega/fast mix in 4.1.15 and that appears to fix the issue (current usage in suspend was lower and resumed properly, but not a real solution).  Anyway I am going to keep working at this, but again I was wondering if anyone else had already encountered and fixed this problem.  It is really noticeable if there is an encrypted file system partition and you attempt to access it after resuming.  Also if you do the old 3.14.52 workaround, the resume code in 4.1.15 complains/warns about an "Unbalanced IRQ wake disable".

Also note that when you dump /proc/interrupts in 4.1.15 that the only GIC interrupts are now IRQ 309 2101000.jr0, IRQ 310 2102000.jr1, and IRQ 311 PCIe PME (almost all the rest are now GPC).  In 3.14.52 almost all the interrupts were GIC (including those listed above,  IRQ 137 2101000.jr0, IRQ 138 2102000.jr1, and IRQ 155 PCIe PME) and none were GPC.  Looking at the IRQ masking registers 1-4 it looks like interrupts with IRQ numbers above 159 are not represented (maybe that is why the workaround no longer works because the IRQ is 309).

Labels (2)
6 Replies

425 Views
daveebright
Contributor III

Carlos,

The following are the patches that work for me while I am waiting for a more official response.  These get around the issue I was having and no longer require the workaround of:

echo enabled > /sys/bus/platform/devices/2100000.aips-bus/2100000.caam/2101000.jr0/power/wakeup"

If you only use the device tree patch, then the workaround is still required.

The call imx_gpc_mf_request_on(jrpriv->irq, 1); in jr.c caam_jr_suspend() sets the bit in the gpc_mf_request_on[] in arch/arm/mach-imx/gpc.c so that when imx_gpc_mix_off() is called during the suspend, then this function returns before turning off mega/fast mix (just like it used to in 3.14.52 with the workaround due to the wake IRQ being enabled).  In 3.14.52 never saw the "Turn off M/F mix!" message in the system logs after the suspend/resume with the workaround, but in 4.1.15 without my patches you always see this message indicating the mega/fast mix was turned off (thus causing the problem where a reboot is needed to recover).  The call to imx_gpc_mf_request_on(jrpriv->irq, 0); in jr.c caam_jr_resume() clears the bit.

Dave

--- linux-rel_imx_4.1.15_1.0.0_ga/arch/arm/boot/dts/imx6sx.dtsi.orig 2016-04-01 14:09:50.000000000 -0400

+++ linux-rel_imx_4.1.15_1.0.0_ga/arch/arm/boot/dts/imx6sx.dtsi 2016-04-01 14:10:34.000000000 -0400

@@ -928,14 +928,14 @@

                     sec_jr0: jr0@1000 {

                           compatible = "fsl,sec-v4.0-job-ring";

                           reg = <0x1000 0x1000>;

-                          interrupt-parent = <&intc>;

+                          interrupt-parent = <&gpc>;

                           interrupts = <0 105 0x4>;

                     };

                     sec_jr1: jr1@2000 {

                           compatible = "fsl,sec-v4.0-job-ring";

                           reg = <0x2000 0x1000>;

-                          interrupt-parent = <&intc>;

+                          interrupt-parent = <&gpc>;

                           interrupts = <0 106 0x4>;

                     };

                };

--- linux-rel_imx_4.1.15_1.0.0_ga/drivers/crypto/caam/jr.c.orig 2016-04-01 12:34:12.000000000 -0400

+++ linux-rel_imx_4.1.15_1.0.0_ga/drivers/crypto/caam/jr.c 2016-04-01 15:09:58.000000000 -0400

@@ -7,6 +7,7 @@

#include <linux/of_irq.h>

#include <linux/of_address.h>

+#include <linux/imx_gpc.h>

#include "compat.h"

#include "regs.h"

@@ -543,6 +544,7 @@ static int caam_jr_suspend(struct device

     struct platform_device *pdev = to_platform_device(dev);

     struct caam_drv_private_jr *jrpriv = platform_get_drvdata(pdev);

+    imx_gpc_mf_request_on(jrpriv->irq, 1);

     if (device_may_wakeup(&pdev->dev))

           enable_irq_wake(jrpriv->irq);

@@ -554,6 +556,7 @@ static int caam_jr_resume(struct device

     struct platform_device *pdev = to_platform_device(dev);

     struct caam_drv_private_jr *jrpriv = platform_get_drvdata(pdev);

+    imx_gpc_mf_request_on(jrpriv->irq, 0);

     if (device_may_wakeup(&pdev->dev))

           disable_irq_wake(jrpriv->irq);

0 Kudos

425 Views
HectorPalacios
Senior Contributor I

Hello,

Is there an official patch for this? I'm experiencing a similar problem in the i.MX6UL in kernel v4.1.

The workaround of doing echo enabled > /sys/bus/platform/devices/2100000.aips-bus/2140000.caam/2141000.jr0/power/wakeup solves the issue but Dave's patch doesn't (at least not my problem on the i,MX6UL).

Thanks,

--

Héctor

0 Kudos

425 Views
CarlosCasillas
NXP Employee
NXP Employee

Hi Dave,

We are internally reviewing your case. In the meantime, you could post any additional information that you could consider useful.

Hope this will be useful for you.

Best regards!

/Carlos

0 Kudos

425 Views
daveebright
Contributor III

Given that in the device tree (imx6xsx.dtsi) for 4.1.15 the gpc entry still contains the line:

fsl,mf-mix-wakeup-irq = <0x7c00000 0x3d00 0x0 0x400240>;

where this represents all the MEGA domain wake IRQs which includes the CAAM job ring 0 interrupt:

0x400240

          ^

This bit is for CAAM job ring 0

However now with the 4.1.15 modifications, this interrupt was moved to the GIC domain which has no facility for wakeups.  So this really breaks the mf-mix-wakeup-irq functionality.  Therefore I suggest modifying the device tree entry minimally for sec_jr0 to change the interrupt-parent = <&intc>; to interrupt-parent = <&gpc>;.  Thus now this interrupt will now be back in the GPC domain and the previous workaround will function properly. Could also make this change for sec_jr1, but currently the mf-mix-wakeup-irq mask does not reference it.

While I was looking into this, I also saw in drivers/crypto/caam/jr.c there are suspend and resume functions.  It may be a good idea to add calls to imx_gpc_mf_request_on() to these functions which I believe could avoid the need for the workaround.

0 Kudos

425 Views
daveebright
Contributor III

After studying the modifications some more in 4.1.15 new code was added to translate all of the GPC interrupt values.  For instance take for example the rtc alarm:

3.14.52 - 51:          0       GIC  51  rtc alarm

4.1.15 - 277:          0       GPC  19 Level     rtc alarm

The GIC has the value 51 and to translate it to GPC simply subtract 32 which gets you to 19.  This translation in 4.1.15 is done for all GPC domain interrupts, but not for the GIC domain.  Thus this is why the work around no longer works.

0 Kudos

425 Views
daveebright
Contributor III

The following is some additional information which shows the contents of /proc/interrupts for each of the kernel versions.  Notice that in the 3.14.52 kernel that the interrupt numbers correspond to the numbers described in the i.MX 6SoloX Applications Processor Reference Manual Chapter 3 Interrupts and DMA Events section 3.2 A.9 Interrupts (in the range from 32 to 159).  For the 4.1.15 kernel this is no longer the case.  I'm not sure yet how much this matters, but it is quite different.  Like I previously said I believe that now for the 4.1.15 interrupts that only the GPC interrupts have the wake-up facility and GIC interrupts do not have the wake-up facility.  That is why the previous work around no longer works.

Maybe it would make sense in the gpc.c file to check if CAAM is enabled (if that is possible) before changing/powering off the Mega/Fast Mix?

3.14.52 Kernel Interrupts

cat /proc/interrupts

            CPU0      

34:          2       GIC  34  sdma

36:          0       GIC  36  20cc000.snvs-pwrkey

38:          3       GIC  38  2224000.lcdif

39:          0       GIC  39  csi

40:          0       GIC  40  pxp-dmaengine

51:          0       GIC  51  rtc alarm

52:          0       GIC  52  snvs-secvio

55:         52       GIC  55  mmc1

56:       2276       GIC  56  mmc2

58:        118       GIC  58  2020000.serial

62:          0       GIC  62  21f4000.serial

67:        115       GIC  67  21f8000.i2c

68:        251       GIC  68  21a0000.i2c

69:         87       GIC  69  21a4000.i2c

70:         86       GIC  70  21a8000.i2c

73:          0       GIC  73  csi

74:          0       GIC  74  2184200.usb

75:          0       GIC  75  2184000.usb

79:          0       GIC  79  202c000.ssi

81:          0       GIC  81  imx_thermal

82:          0       GIC  82  asrc

87:       2713       GIC  87  i.MX Timer Tick

112:          0       GIC 112  20bc000.wdog

122:          0       GIC 122  imx-mu

132:          1       GIC 132  2280000.adc

133:          1       GIC 133  2284000.adc

134:          0       GIC 134  21b4000.ethernet

135:          0       GIC 135  21b4000.ethernet

137:        264       GIC 137  2101000.jr0

138:       3534       GIC 138  2102000.jr1

141:          8       GIC 141  21e4000.qspi

148:          0       GIC 148  imx6sx-sema4

150:        541       GIC 150  2188000.ethernet

151:          0       GIC 151  2188000.ethernet

155:          0       GIC 155  PCIe PME, ath9k

156:          0       GIC 156  220c000.dcic

157:          0       GIC 157  2210000.dcic

177:          0  gpio-mxc  17  headphone detect

178:          0  gpio-mxc  18  Volume Up

179:          0  gpio-mxc  19  Volume Down

202:          0  gpio-mxc  10  2198000.usdhc cd

325:          0  gpio-mxc   5  isl29023, mag3110

334:          0  gpio-mxc  14  abx8xx

IPI0:          0  CPU wakeup interrupts

IPI1:          0  Timer broadcast interrupts

IPI2:          0  Rescheduling interrupts

IPI3:          0  Function call interrupts

IPI4:          0  Single function call interrupts

IPI5:          0  CPU stop interrupts

IPI6:        148  IRQ work interrupts

IPI7:          0  completion interrupts

Err:          0

4.1.15 Kernel Interrupts

cat /proc/interrupts

           CPU0      

16:   45688372       GPC  55 Level     i.MX Timer Tick

20:          0       GPC  20 Level     snvs-secvio

22:        533       GPC  26 Level     2020000.serial

23:          0       GPC  47 Level     202c000.ssi

24:          0       GPC  50 Level     2034000.asrc

50:          0  gpio-mxc  17 Edge      headphone detect

51:          0  gpio-mxc  18 Edge      Volume Up

52:          0  gpio-mxc  19 Edge      Volume Down

77:          0  gpio-mxc  10 Edge      2198000.usdhc cd

208:          0  gpio-mxc   5 Edge      isl29023, mag3110

217:          0  gpio-mxc  14 Edge      rtc-abx80x

269:          0       GPC  80 Level     20bc000.wdog

272:          0       GPC  49 Level     imx_thermal

277:          0       GPC  19 Level     rtc alarm

278:          2       GPC   4 Level     20cc000.snvs:snvs-powerkey

282:     261922       GPC   2 Level     sdma

283:          0       GPC  43 Level     2184000.usb

284:          0       GPC  42 Level     2184200.usb

285:     338974       GPC 118 Level     2188000.ethernet

286:          0       GPC 119 Level     2188000.ethernet

287:         52       GPC  23 Level     mmc1

288:    2095779       GPC  24 Level     mmc2

289:     468174       GPC  36 Level     21a0000.i2c

290:        109       GPC  37 Level     21a4000.i2c

291:        116       GPC  38 Level     21a8000.i2c

292:          0       GPC 102 Level     21b4000.ethernet

293:          0       GPC 103 Level     21b4000.ethernet

295:         18       GPC 109 Level     21e4000.qspi

296:          2       GPC  30 Level     21f4000.serial

297:        115       GPC  35 Level     21f8000.i2c

298:          0       GPC   7 Level     csi

299:          0       GPC 124 Level     220c000.dcic

300:          0       GPC 125 Level     2210000.dcic

301:          0       GPC   8 Level     pxp-dmaengine

302:          0       GPC  41 Level     csi

303:          6       GPC   6 Level     2224000.lcdif

304:          1       GPC 100 Level     2280000.adc

305:          1       GPC 101 Level     2284000.adc

306:          0       GPC 116 Level     imx6sx-sema4

307:          0       GPC  90 Level     imx-mu

309:        264       GIC 137 Level     2101000.jr0

310:   52326450       GIC 138 Level     2102000.jr1

311:    3861065       GIC 155 Level     PCIe PME, ath9k

IPI0:          0  CPU wakeup interrupts

IPI1:          0  Timer broadcast interrupts

IPI2:          0  Rescheduling interrupts

IPI3:          0  Function call interrupts

IPI4:          0  Single function call interrupts

IPI5:          0  CPU stop interrupts

IPI6:          0  IRQ work interrupts

IPI7:          0  completion interrupts

Err:          0

0 Kudos