SW workarounds used to fix ERR005723 in Linux BSP(private)

Document created by Richard Zhu Employee on Jul 15, 2013Last modified by Richard Zhu Employee on Jul 16, 2013
Version 1Show Document
  • View in full screen mode

ERR005723           PCIe: PCIe does not support L2 Power Down

 

Description:

When PCIe works as Root Complex, it can exit L2 mode only through reset. Since PCIe doesn't have a dedicated reset control bit, it cannot exit L2 mode.

 

Projected Impact:

PCIe does not support L2 Power Down

 

Workarounds:

The PCIe can be put in PDDQ mode to save on PCIe PHY power and wakeup only by the OOB (Out of Band) wakeup signal (since wakeup by a beacon from link partner is not supported) driven from the link partner (End Point). This signal could be used as a GPIO interrupt to exit this mode.

The limitation of this workaround is that the link partner cannot be put into L2.

 

Proposed Solution:

                No fix scheduled

 

Linux BSP Status:

                No software workaround available

 

SW workaround used to fix ERR005723 in Linux BSP

Why the original workarounds can’t be implemented in Linux BSP

* PCIe controller doesn’t have the reset mechanism that can be used when re-insmod the PCIe driver without power down/up PCIe module.

* During the PCie driver rmmod/insmod operations, the PCIe CLKs would be turned off/on. IC can’t guarantee that the PCIe PHY can work well and re-establish the PCIe link properly.

One SIMPLE SW workaround for this errata

imx: pcie: toggle bit18 of grp1 fix pcie can't exit L2 issue.

 

Set bit18 of gpr1 before enter into supend, and clean it

after resume, can fix the following errata.

Errata ERR005723_PCIe PCIe does not support L2 Power Down.

About the details, please refer to the attached patch.

"0001-imx-pcie-toggle-bit18-of-grp1-fix-pcie-can-t-exit-L2.patch"

 


The conception of the other SW workaround (System warm-reset)

The procedures of the original suspend/resume.

Suspend

  1. User suspend command echo mem > /sys/power/state
  2. All driver call suspend function
  3. SRPG,  ARM save all state to memory
  4. Enter Stop mode and Power down ARM

Resume:

  1. GPC receive IRQ
  2. Wake up system
  3. Power on ARM domain.
  4. ROM code running
  5. Jump to SRPG point
  6. Recovery ARM status from memory
  7. Call all devices resume function.

Because PCIe only reset by system reset, we need change above follow.

Resume:

  1. GPC receive IRQ
  2. Wake up system
  3. Power on ARM domain.
  4. ROM code running
  5. Jump to SRPG point
  6. Warm Reset system, memory context will be kept. But all peripheral status lost.
  7. ROM code running
  8. Jump to SRPG point again.
  9. Recovery ARM status from memory
  10. Call all devices resume function.
  11. Resume function call init to initialize it.  And recover to the status saved before.

Impact:

  1. Can’t support usb remote wake up, which required 4ms responsive
  2. Longer latency, warm reset need some ms.  The recovery of the device status needs some more ms.

Risk:

  1. Current BSP have not tested above follow
  2. Device driver have not supported this follow yet. Need additional work to enable \debug\test it.

Modules enabled in this workaround now:

* UART* ENET* PCIe

Tests procedure.

HW:

one i.MX6Q SD boards, and one INTEL pciex1 1000M CT network card.2013-05-21_17-48-03_486.jpg

SW(The images used by me are attached):

* Apply the attached patches(kernel and uboot) to the kernel/uboot source codes, re-build, get the images.

Kernel is based on imx_3.0.35_4.0 release, uboot , is based on imx_v2009.08

# build out SD/MMC and USB driver to make DRAM hibernate work

usb_sd_build_out.png

# build pcie in.

pcie_build_in.png

*procedure of the suspend/resume tests;

    # unload ep's driver --> suspend/resume --> reload ep's driver.

NOTE:

Please make sure that the command line contains “no_console_suspend”The command used to enable the console input wake up after login the consol:echo enabled > /sys/devices/platform/imx-uart.0/tty/ttymxc0/power/wakeup

Log when the INTEL CT 1G network card is used:

-------------------------------log--------------------------------------------PM: Syncing filesystems ... done.                                            start suspendFreezing user space processes ... (elapsed 0.01 seconds) done.Freezing remaining freezable tasks ... (elapsed 0.01 seconds) done.add wake up source irq 101add wake up source irq 99add wake up source irq 103add wake up source irq 51add wake up source irq 58PM: suspend of devices complete after 15.482 msecsPM: late suspend of devices complete after 0.823 msecsDisabling non-boot CPUs ...CPU1: shutdownCPU2: shutdownCPU3: shutdownIMX PCIe imx_pcie_pltfm_suspend entering.IMX PCIe imx_pcie_pltfm_suspend exit.          suspendedU-Boot 2009.08-00679-g6ec6783 (May 20 2013 - 14:50:20)     resumeCPU: Freescale i.MX6 family TO1.2 at 792 MHzsrc 0x92eac8resume 0x92eac8jump to resumeIMX PCIe imx_pcie_pltfm_resume entering.IMX PCIe imx_pcie_pltfm_resume pcie start re-link.IMX PCIe port imx_pcie_pltfm_resume: re-link up.Enabling non-boot CPUs ...CPU1: Booted secondary processorCalibrating delay loop (skipped) already calibrated this CPU

  1. i.MXC CPU frequency driver

CPU1 is upCPU2: Booted secondary processorCalibrating delay loop (skipped) already calibrated this CPU

  1. i.MXC CPU frequency driver

CPU2 is upCPU3: Booted secondary processorCalibrating delay loop (skipped) already calibrated this CPU

  1. i.MXC CPU frequency driver

CPU3 is up

PM: early resume of devices complete after 0.974 msecs

remove wake up source irq 58

imx-ipuv3 imx-ipuv3.0: IPU DMFC DP HIGH RESOLUTION: 1(0,1), 5B(2~5), 5F(6,7)

imx-ipuv3 imx-ipuv3.1: IPU DMFC DP HIGH RESOLUTION: 1(0,1), 5B(2~5), 5F(6,7)

remove wake up source irq 51

remove wake up source irq 103

remove wake up source irq 101

remove wake up source irq 99

PM: resume of devices complete after 54.174 msecs

Restarting tasks ... done.

PHY: 1:01 - Link is Up - 100/Full                            resume is ok, reload ep’s driver

num is 61

e1000e: Intel(R) PRO/1000 Network Driver - 1.3.10-k2

e1000e: Copyright(c) 1999 - 2011 Intel Corporation.

e1000e 0000:01:00.0: Disabling ASPM L0s

e1000e 0000:01:00.0: (unregistered net_device): Failed to initialize MSI-X interrupts.  Falling back to MSI interrupts.

e1000e 0000:01:00.0: (unregistered net_device): Failed to initialize MSI interrupts.  Falling back to legacy interrupts.

e1000e 0000:01:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:1b:21:3a:18:8b

e1000e 0000:01:00.0: eth1: Intel(R) PRO/1000 Network Connection

e1000e 0000:01:00.0: eth1: MAC: 3, PHY: 8, PBA No: E42641-005

e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

PING 192.168.0.1 (192.168.0.1): 56 data bytes

64 bytes from 192.168.0.1: seq=0 ttl=64 time=3.126 ms

64 bytes from 192.168.0.1: seq=1 ttl=64 time=0.244 ms

64 bytes from 192.168.0.1: seq=2 ttl=64 time=0.232 ms

64 bytes from 192.168.0.1: seq=3 ttl=64 time=0.206 ms

64 bytes from 192.168.0.1: seq=4 ttl=64 time=0.222 ms

64 bytes from 192.168.0.1: seq=5 ttl=64 time=0.207 ms

64 bytes from 192.168.0.1: seq=6 ttl=64 time=0.250 ms

64 bytes from 192.168.0.1: seq=7 ttl=64 time=0.209 ms

64 bytes from 192.168.0.1: seq=8 ttl=64 time=0.154 ms

64 bytes from 192.168.0.1: seq=9 ttl=64 time=0.211 ms

 

--- 192.168.0.1 ping statistics ---

10 packets transmitted, 10 packets received, 0% packet loss

round-trip min/avg/max = 0.154/0.506/3.126 ms

PM: Syncing filesystems ... done.                                   ep’s functions are ok, re-do the suspend/resume tests

Freezing user space processes ... (elapsed 0.01 seconds) done.

-------------------------------end--------------------------------------------



Outcomes