System will not resume from suspend to memory with PCIE enabled

[scottwarner]

This is on an i.MX6Solo custom board based on the sabresd, running the yocto dora 3.10.9_alpha kernel.  Suspend/Resume from suspend to memory was working fine until PCIE was enabled.  With PCIE enabled most times the resume fails during the noirq resume of device, in the pcie driver.  It will sometimes wake, but most times it hangs at the first read of the pcie's own config space as part of the pci_power_up call where it's updating the current state, it's trying to read PM_CTRL register to update the power state.  I've tested the same thing on the sabresd board and it runs fine there, it also ran fine on the custom board with the 3.0.35 kernel.  The problem occurs with and without a card plugged into the pcie slot.  Any ideas on what may cause this?

Thanks,
Scott

[AnsonHuang]

Hi, Scott

     We have a patch to enable PCIe PM in 3.10.9 Beta release, please get it to have a try.

commit 615ee93d9fcafc55ee487588fa1c39be47bc3d54

Author: Richard Zhu <r65037@freescale.com>

Date:   Mon Nov 18 16:42:18 2013 +0800

    ENGR00288409 pcie: enable pcie pm when pcie is enabled

    L2 can exit by 'reset' or Inband beacon (from remote EP)

    toggling phy_powerdown has same effect as 'inband beacon'

    So, toggle bit18 of GPR1, used as a workaround of errata

    "PCIe PCIe does not support L2 Power Down"

    WARNING: This is not official workaround for ERR005723. But we

    have not found issue yet. User should take own risk to use it.

    Signed-off-by: Richard Zhu <r65037@freescale.com>

arch/arm/mach-imx/pm-imx6.c

diff --git a/arch/arm/mach-imx/pm-imx6.c b/arch/arm/mach-imx/pm-imx6.c

index 5629f11..a4af0c5 100644

--- a/arch/arm/mach-imx/pm-imx6.c

+++ b/arch/arm/mach-imx/pm-imx6.c

@@ -18,6 +18,9 @@

#include <linux/of_irq.h>

#include <linux/suspend.h>

#include <linux/genalloc.h>

+#include <linux/regmap.h>

+#include <linux/mfd/syscon.h>

+#include <linux/mfd/syscon/imx6q-iomuxc-gpr.h>

#include <asm/cacheflush.h>

#include <asm/fncpy.h>

#include <asm/proc-fns.h>

@@ -223,6 +226,24 @@ static int imx6_suspend_finish(unsigned long val)

static int imx6_pm_enter(suspend_state_t state)

{

+       struct regmap *g;

+

+       /*

+       /*

+        * L2 can exit by 'reset' or Inband beacon (from remote EP)

+        * toggling phy_powerdown has same effect as 'inband beacon'

+        * So, toggle bit18 of GPR1, used as a workaround of errata

+        * "PCIe PCIe does not support L2 Power Down"

+        */

+       if (IS_ENABLED(CONFIG_PCI_IMX6)) {

+               g = syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");

+               if (IS_ERR(g)) {

+                       pr_err("failed to find fsl,imx6q-iomux-gpr regmap\n");

+                       return PTR_ERR(g);

+               }

+               regmap_update_bits(g, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD,

+                               IMX6Q_GPR1_PCIE_TEST_PD);

+       }

+

        switch (state) {

        case PM_SUSPEND_STANDBY:

                imx6_set_lpm(STOP_POWER_ON);

@@ -259,6 +280,17 @@ static int imx6_pm_enter(suspend_state_t state)

                return -EINVAL;

        }

+       /*

+        * L2 can exit by 'reset' or Inband beacon (from remote EP)

+        * toggling phy_powerdown has same effect as 'inband beacon'

+        * So, toggle bit18 of GPR1, used as a workaround of errata

+        * "PCIe PCIe does not support L2 Power Down"

+        */

+       if (IS_ENABLED(CONFIG_PCI_IMX6)) {

+               regmap_update_bits(g, IOMUXC_GPR1, IMX6Q_GPR1_PCIE_TEST_PD,

+                               !IMX6Q_GPR1_PCIE_TEST_PD);

+       }

+

        return 0;

}

[scottwarner]

Hi Youngcai,

Unfortunately I already had that patch and it didn't help with this problem, although it did fix the issue we had on the linux_3.0.35 bsp when trying to wake from sleep with a wifi card plugged into the pcie. 

This current issue happens whether or not the card is plugged into the pcie.  When I initially ported the 3.10.9_alpha bsp I had to change the pcie reference clock from the 125MHz clock from the sabresd dtb file to 100MHz (sata_ref_100m 187) for our board.  Prior to making that change pcie would not even linkup.  After making that change it linked and detected the wifi card, wifi card works fine.  We're just left with this new wakeup issue, happens with or without the card.

Thanks,

Scott

[michaelrobbelot]

Scott,

I'm testing to see if adding a 1 ms per device delay w/ udelay in drivers/base/power/main.c device_resume_noirq (dev, state) at the end of the function will correct the problem. So far, so good; though, it's a pretty harsh workaround/fix to prevent a lockup on resume. I'm hoping someone comes up with something better.

Michael Robbeloth

[wallyyeh]

Hi, Michael:

   hi, does this workaround working for 3.0.35 kernel? I tried add udelay(1000) under error = device_resume_noirq(dev, state) in function "dpm_resume_noirq(pm_message_t state)"

and system will crash.

[michaelrobbelot]

Wally,

We found this workaround led to device instability as you have seen. We finally decided to just add a partial wakelock to the launcher app and be done with it, which is okay for our device since it is AC connected 100% of the time. It's a shame, though, I would have liked to have found a solution that worked.-- this workaround bypasses power management entirely.

Michael Robbeloth

[wallyyeh]

I appreciate your reply :smileyhappy:, and I think your partial wakelock idea is good.

by test, close peripheral device the power is just 2.3Watt, against real suspend is 1.4Watt. that's acceptable to me.