Power Management issues in MPC8313 processor

[sricanchivaram]

Hi,

I have a modem with a Freescale MPC8313E processor. I am trying to enable power savings in the processor by placing it in Standby mode and resume normal operation with a Wake-On-LAN magic packet.

Following the directions in the Freescale Power Management app note, I enabled Power Management Support in the Linux kernel and device tree configurations. 

I ran the following command on the board:

echo standby > /sys/power/state

This caused the console to hang and there was no further response to keyboard inputs. I enabled ‘no_console_suspend’ in the kernel and when I loaded the new build and enabled standby mode, I observed an Oops trace:

RASCOM_QCU.7.0.0013 $ echo standby > /sys/power/state

<6>PM: Syncing fFreezing user space processes ... ilesystems ... <7>PM: Entering standby sleep

Unable to handle kernel paging request for instruction fetch

Faulting instruction address: 0x616d6570

Oops: Kernel access of bad area, sig: 11 [#1]

MPC831x RDB

Modules linked in: dsp rcspi modem i2c_mpc thermal_sys lm92 hwmon [last unloaded: modem]

NIP: 616d6570 LR: c0165224 CTR: 616d6573

REGS: cd087d30 TRAP: 0400 Not tainted  (2.6.27)

MSR: 20001032 <ME,IR,DR>  CR: 28002024 XER: 20000000

TASK = cc312400[1196] 'echo' THREAD: cd086000

GPR00: 00000002 cd087de0 cc312400 cf821800 cd087de8 00000002 c06e0000 c06da4a0

GPR08: c06da948 616d6573 00003fff c06c6308 28002022 10091248 0fffc000 100050b8

GPR16: 1008a270 10068810 100687c8 10068814 00000000 1008c284 1008c294 c0246180

GPR24: c02ab9e4 c02ab9dc c06cc4f4 00000006 cd087e08 00000002 c06c595c cf821808

NIP [616d6570] 0x616d6570

LR [c0165224] platform_pm_suspend_noirq+0x84/0x88

Call Trace:

[cd087de0] [c0167e6c] pm_dev_dbg+0x70/0x18c (unreliable)

[cd087df0] [c0167cb4] pm_noirq_op+0x58/0xc8

[cd087e00] [c0168e08] device_power_down+0x78/0x1c4

[cd087e40] [c00471bc] suspend_devices_and_enter+0x1ec/0x258

[cd087e70] [c00473ac] enter_state+0x13c/0x198

[cd087e90] [c00474bc] state_store+0xb4/0xf8

[cd087eb0] [c013af68] kobj_attr_store+0x24/0x3c

[cd087ec0] [c00bf734] sysfs_write_file+0xe8/0x1e8

[cd087ef0] [c0078bbc] vfs_write+0xb4/0x16c

[cd087f10] [c0078d7c] sys_write+0x4c/0xa4

[cd087f40] [c000fa88] ret_from_syscall+0x0/0x38

--- Exception: c01 at 0x4802679c

    LR = 0x4803f0e8

Instruction dump:

XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX

XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX

---[ end trace b6a2457eea68b760 ]---

1. done.

<7>PM: Preparing system for standby sleep

<1>Freezing user space processes ... <7>PM: Entering standby sleep

<7>platform mpc83xx_wdt.0: preparing suspend

<7>fsl-gianfar fsl-gianfar.0: preparing suspend

<7>fsl-gianfar fsl-gianfar.1: preparing suspend

<7>fsl-gianfar_mdio fsl-gianfar_mdio.14: preparing suspend

<7>serial8250 serial8250.0: preparing suspend

<7>serial8250 serial8250: preparing suspend

<7>platform Fixed MDIO bus.0: preparing suspend

<7>lm92 0-0049: legacy suspend

<7>lm92 0-0048: legacy suspend

<7>platform Fixed MDIO bus.0: suspend

<7>mdio_bus 24520:1f: legacy suspend

<7>Generic PHY 24520:01: legacy suspend

<7>Generic PHY 24520:00: legacy suspend

<7>serial8250 serial8250: suspend

<7>of_platform ee000600.timer: legacy suspend

<7>of_platform ee000500.timer: legacy suspend

<7>mpc83xx-pmc ee000b00.power: legacy suspend

<7>of_platform ee000700.pic: legacy suspend

<7>of_platform ee030000.crypto: legacy suspend

<7>of_platform ee004600.serial: legacy suspend

<7>of_platform ee004500.serial: legacy suspend

<7>of_platform ee025000.ethernet: legacy suspend

<7>of_platform ee024000.ethernet: legacy suspend

<7>of_platform ee024520.mdio: legacy suspend

<7>of_platform ee024e00.ptimer: legacy suspend

<7>of_platform ee0082a8.dma: legacy suspend

<7>mpc-i2c ee003100.i2c: legacy suspend

<7>mpc-i2c ee003000.i2c: legacy suspend

<7>of_platform ee000200.wdt: legacy suspend

<7>of_platform ee000000.soc8313: legacy suspend

<7>of-flash fc000000.flash: legacy suspend

<7>of_platform ee005000.localbus: legacy suspend

<7>serial8250 serial8250.0: suspend

<7>fsl-gianfar_mdio fsl-gianfar_mdio.14: suspend

<7>fsl-gianfar fsl-gianfar.1: suspend

<7>fsl-gianfar fsl-gianfar.0: suspend

<7>platform mpc83xx_wdt.0: suspend

<6>PM: suspend devices took 0.000 seconds

<7>platform Fixed MDIO bus.0: LATE suspend

<7>serial8250 serial8250: LATE suspend

<7>serial8250 serial8250.0: LATE suspend

<7>fsl-gianfar_mdio fsl-gianfar_mdio.14: LATE suspend

<1>Unable to handle kernel paging request for instruction fetch

<1>Faulting instruction address: 0x616d6570

<1>Oops: Kernel access of bad area, sig: 11 [#1]

<1>MPC831x RDB

<1>Modules linked in: dsp rcspi modem i2c_mpc thermal_sys lm92 hwmon [last unloaded: modem]

<1>NIP: 616d6570 LR: c0165224 CTR: 616d6573

<1>REGS: cd087d30 TRAP: 0400   Not tainted (2.6.27)

<1>MSR: 20001032 <ME,IR,DR>  CR: 28002024 XER: 20000000

<1>TASK = cc312400[1196] 'echo' THREAD: cd086000

<6>GPR00: 00000002 cd087de0 cc312400 cf821800 cd087de8 00000002 c06e0000 c06da4a0

<6>GPR08: c06da948 616d6573 00003fff c06c6308 28002022 10091248 0fffc000 100050b8

<6>GPR16: 1008a270 10068810 100687c8 10068814 00000000 1008c284 1008c294 c0246180

<6>GPR24: c02ab9e4 c02ab9dc c06cc4f4 00000006 cd087e08 00000002 c06c595c cf821808

<1>NIP [616d6570] 0x616d6570

<1>LR [c0165224] platform_pm_suspend_noirq+0x84/0x88

<1>Call Trace:

<1>[cd087de0] [c0167e6c] pm_dev_dbg+0x70/0x18c (unreliable)

<1>[cd087df0] [c0167cb4] pm_noirq_op+0x58/0xc8

<1>[cd087e00] [c0168e08] device_power_down+0x78/0x1c4

<1>[cd087e40] [c00471bc] suspend_devices_and_enter+0x1ec/0x258

<1>[cd087e70] [c00473ac] enter_state+0x13c/0x198

<1>[cd087e90] [c00474bc] state_store+0xb4/0xf8

<1>[cd087eb0] [c013af68] kobj_attr_store+0x24/0x3c

<1>[cd087ec0] [c00bf734] sysfs_write_file+0xe8/0x1e8

<1>[cd087ef0] [c0078bbc] vfs_write+0xb4/0x16c

<1>[cd087f10] [c0078d7c] sys_write+0x4c/0xa4

<1>[cd087f40] [c000fa88] ret_from_syscall+0x0/0x38

<1>--- Exception: c01 at 0x4802679c

<1>    LR = 0x4803f0e8

<1>Instruction dump:

<1>XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX

<1>XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX

<4>---[ end trace b6a2457eea68b760 ]---

Segmentation fault

I tried commenting out the kernel code in drivers/base/platform.c -> platform_legacy_suspend_late() function:

/*     if (dev->driver && drv->suspend_late)

       { } */

This prevents the segmentation fault but it causes the resume functions for all the peripherals to be called as soon as the suspend operations are complete i.e. the processor goes into standby and then immediately goes back to normal operation.

I found another thread that dealt with Power Management issues on the Freescale MPC8313 processor:

https://lists.ozlabs.org/pipermail/linuxppc-dev/2012-January/095240.html

The resolution of this issue seems to be related to the JTAG TRST pin being disabled. This is not relevant in my case as the TRST on my board is already inactive.

Any advice or suggestions are most welcome.

Thanks,

Srivatsan

[scottwood]

Which kernel version are you using?

[sricanchivaram]

Hello Scott,

Thank you for responding.

I am using kernel version 2.6.27.

I made more progress on this issue.

As mentioned in my original posting, suspending the processor results in a segmentation fault. This seems to happen in the kernel code in drivers/base/platform.c -> platform_legacy_suspend_late() function. The following driver function call is the offending code:

ret =  drv->suspend_late(pdev, mesg);

When I commented out this line, the processor seemed to go into suspend and then resume almost immediately. This was happening because of an on-board peripheral that was sending a periodic interrupt to the processor. Once I disabled that device, I could see that the processor was placed in standby mode. The processor was able to resume successfully with a wake-on-lan packet sent from another terminal.

So as of now, I am able to put the processor in standby with the above function call commented out. The segmentation fault occurs when the suspend_late function is called for fsl-gianfar_mdio (as per the debug log). The driver function call is in an if-block:

   if (dev->driver && drv->suspend_late)

   {

     ret =  drv->suspend_late(pdev, mesg);

    }

The if-block is true only for fsl-gianfar_mdio and false for all the other peripherals that are suspending. I am not sure why a bogus function pointer address (0x616d6573) is being referenced there.

Your comments/suggestions are most welcome.

Thanks,

Srivatsan

[scottwood]

Ah, I now see that I already responded on an external mailing list back in October. :-)

I think the problem may be due to gianfar_mii.c using driver_register() rather than platform_driver_register() -- and thus declaring only a struct device_driver rather than a struct platform_device_driver.  This was fixed in v2.6.29.

[sricanchivaram]

Without this patch in 2.6.27, commenting out the drv->suspend_late() call seems to be the solution I have at the moment. Please let me know if you think this is reasonable or if this is not a good idea.

At this stage of our product, I would like to avoid changing the kernel version.

[scottwood]

You could try backporting just that patch that converted gianfar_mii into an of_platform driver, or you could change gianfar_mii yourself to use the proper platform driver struct and registration function.

I do not recommend just commenting out that line.