Kernel panic in AOSP after switching to mode "Suspend-to-RAM"

[RKhairulin]

I'm using qmmek like board with  imx8qm processor. I sent the system to the "Suspend-to-RAM" mode by command "echo mem > /sys/power/state".  Then I pushed a button, which reset the processor using pin ON_OFF_BUTTON (BE47).

Power key have dedicated interrupt as the wakeup source:

sc_pwrkey: sc-powerkey {
                compatible = "fsl,imx8-pwrkey";
                linux,keycode = <KEY_POWER>;
                wakeup-source;
};

When the system was exiting from "Suspend-to-RAM" mode the kernel panic error occurred:

[ 4355.463991] Enabling non-boot CPUs ...
[ 4355.464928] Detected PIPT I-cache on CPU1
[ 4355.464953] GICv3: CPU1: found redistributor 101 region 0:0x0000000051ba0000
[ 4355.464977] CPU1: Booted secondary processor 0x0000000101 [0x410fd082]
[ 4355.468283] CPU1 is up
[ 4355.477086] Detected VIPT I-cache on CPU2
[ 4355.477111] GICv3: CPU2: found redistributor 0 region 0:0x0000000051b00000
[ 4355.477157] CPU2: Booted secondary processor 0x0000000000 [0x410fd034]
[ 4355.477171] CPU0: update max cpu_capacity 1024
[ 4355.482065] CPU2 is up
[ 4355.483020] Detected VIPT I-cache on CPU3
[ 4355.483036] GICv3: CPU3: found redistributor 1 region 0:0x0000000051b20000
[ 4355.483055] CPU3: Booted secondary processor 0x0000000001 [0x410fd034]
[ 4355.483219] CPU1: update max cpu_capacity 1024
[ 4355.487218] CPU0: update max cpu_capacity 1024
[ 4355.487345] CPU3 is up
[ 4355.488291] Detected VIPT I-cache on CPU4
[ 4355.488306] GICv3: CPU4: found redistributor 2 region 0:0x0000000051b40000
[ 4355.488327] CPU4: Booted secondary processor 0x0000000002 [0x410fd034]
[ 4355.492659] CPU4 is up
[ 4355.493602] Detected VIPT I-cache on CPU5
[ 4355.493619] GICv3: CPU5: found redistributor 3 region 0:0x0000000051b60000
[ 4355.493639] CPU5: Booted secondary processor 0x0000000003 [0x410fd034]
[ 4355.495219] CPU3: update max cpu_capacity 1024
[ 4355.498129] CPU5 is up
[ 4355.499238] CPU0: update max cpu_capacity 1024
[ 4355.551503] PM: dpm_run_callback(): genpd_resume_noirq+0x0/0xc0 returns -16
[ 4355.551507] PM: Device hdmi_pixel_link_clk failed to resume noirq: error -16
[ 4355.561767] [drm] Started firmware!
[ 4355.561955] [drm:cdns_mhdp_firmware_init_imx8qm] *ERROR* NO HDMI FW running
[ 4355.563637] [drm] Mode: 1920x1080p148500
[ 4355.591415] [drm:cdns_hdmi_phy_set_imx8qm] *ERROR* NO HDMI FW running
[ 4355.591417] [drm] Sink Not Support SCDC
[ 4360.591796] [drm:cdns_hdmi_mode_set] *ERROR* cdns_hdmi_mode_set, ret = -110
[ 4360.737982] imx-cs42888 sound-cs42888: failed to find codec platform device
[ 4360.959150] OOM killer enabled.
[ 4360.962292] Restarting tasks ... 
[ 4360.964878] SError Interrupt on CPU1, code 0xbf000002 -- SError
[ 4360.964879] CPU: 1 PID: 229 Comm: cdns-mhdp-cec Tainted: G           O      5.4.24-00046-g9509ded1da88-dirty #11
[ 4360.964880] Hardware name: Freescale i.MX8QM MEK (DT)
[ 4360.964881] pstate: 40400085 (nZcv daIf +PAN -UAO)
[ 4360.964882] pc : el1_irq+0xb0/0x1c0
[ 4360.964882] lr : cdns_mhdp_bus_read+0x24/0x108
[ 4360.964883] sp : ffff800013423c90
[ 4360.964884] x29: ffff800013423dc0 x28: ffff0008f67c4f80 
[ 4360.964886] x27: 0000000000000001 x26: 0000000000033844 
[ 4360.964888] x25: ffff800010605a58 x24: ffff80001125e6dc 
[ 4360.964889] x23: 0000000060400005 x22: ffff800010600720 
[ 4360.964891] x21: 0000000081104000 x20: 0000ffffffffffff 
[ 4360.964893] x19: ffff0008f752ae60 x18: 0000000000000000 
[ 4360.964895] x17: 00000000000003fc x16: 00000000000002a8 
[ 4360.964896] x15: 000000000000002c x14: 00000000000001d8 
[ 4360.964898] x13: 0000000000000000 x12: ffff80001100d060 
[ 4360.964900] x11: 0000000000000000 x10: 0000000000000000 
[ 4360.964901] x9 : 000000000000094c x8 : ffff80001308d94c 
[ 4360.964903] x7 : 0000000000000000 x6 : 0000000000000000 
[ 4360.964905] x5 : 0000000000000000 x4 : 0000000000000000 
[ 4360.964906] x3 : 0000000000000001 x2 : ffff0008ff37aed0 
[ 4360.964908] x1 : 000000000003394c x0 : ffff0008f752ae60 
[ 4360.964910] Kernel panic - not syncing: Asynchronous SError Interrupt
[ 4360.964912] CPU: 1 PID: 229 Comm: cdns-mhdp-cec Tainted: G           O      5.4.24-00046-g9509ded1da88-dirty #11
[ 4360.964913] Hardware name: Freescale i.MX8QM MEK (DT)
[ 4360.964913] Call trace:
[ 4360.964914]  dump_backtrace+0x0/0x134
[ 4360.964915]  show_stack+0x14/0x1c
[ 4360.964915]  dump_stack+0xa0/0xd8
[ 4360.964916]  panic+0x148/0x338
[ 4360.964917]  panic+0x0/0x338
[ 4360.964918]  arm64_serror_panic+0x70/0x7c
[ 4360.964918]  do_serror+0x160/0x170
[ 4360.964919]  el1_error+0xbc/0x160
[ 4360.964920]  el1_irq+0xb0/0x1c0
[ 4360.964920]  cdns_mhdp_bus_read+0xbc/0x108
[ 4360.964921]  mhdp_cec_poll_worker+0x98/0x238
[ 4360.964922]  kthread+0x10c/0x11c
[ 4360.964922]  ret_from_fork+0x10/0x18
[ 4360.966256] SMP: stopping secondary CPUs
[ 4360.966257] Kernel Offset: disabled
[ 4360.966258] CPU features: 0x0002,2100600c
[ 4360.966259] Memory Limit: none
[ 4360.966260] trusty-log panic notifier - trusty version Built: 15:19:11 Mar 20 2020

Should I add any parameters to the dts file to fix problem?

Thanks!

[RKhairulin]

Anybody are going answer me? We are industrial users of NXP

[Zhiming_Liu]

My test is based on android11_1.0.0,and i can't reproduce the issue.From the log you attached,seems that hdmi cec function can't accept interput so this caused Serror exception.

Do you test suspend to ram many times? Or just one time , then this issue appears.

[  161.897951] misc FreescaleGyroscope: mma enable setting active
[  162.038332] PM: suspend entry (deep)
[  162.049401] Filesystems sync: 0.007 seconds
[  162.053643] Freezing user space processes ... (elapsed 0.002 seconds) done.
[  162.063266] OOM killer disabled.
[  162.066502] Freezing remaining freezable tasks ... (elapsed 0.099 seconds) done.
[  162.173978] printk: Suspending console(s) (use no_console_suspend to debug)
[  162.998158] PM: suspend devices took 0.816 seconds
[  163.029053] Disabling non-boot CPUs ...
[  163.030286] CPU1: shutdown
[  163.031315] psci: CPU1 killed (polled 0 ms)
[  163.033398] CPU2: shutdown
[  163.034423] psci: CPU2 killed (polled 0 ms)
[  163.035761] CPU3: shutdown
[  163.036039] CPU0: update max cpu_capacity 363
[  163.036116] psci: CPU3 killed (polled 4 ms)
[  163.037162] CPU4: shutdown
[  163.038185] psci: CPU4 killed (polled 0 ms)
[  163.039143] CPU5: shutdown
[  163.040039] psci: CPU5 killed (polled 4 ms)
[  163.040951] Enabling non-boot CPUs ...
[  163.041906] Detected VIPT I-cache on CPU1
[  163.041941] GICv3: CPU1: found redistributor 1 region 0:0x0000000051b20000
[  163.041992] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[  163.043110] CPU1 is up
[  163.044058] Detected VIPT I-cache on CPU2
[  163.044075] GICv3: CPU2: found redistributor 2 region 0:0x0000000051b40000
[  163.044100] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[  163.044786] CPU2 is up
[  163.045627] Detected VIPT I-cache on CPU3
[  163.045643] GICv3: CPU3: found redistributor 3 region 0:0x0000000051b60000
[  163.045668] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[  163.046258] CPU3 is up
[  163.048232] Detected PIPT I-cache on CPU4
[  163.048250] GICv3: CPU4: found redistributor 100 region 0:0x0000000051b80000
[  163.048278] CPU4: Booted secondary processor 0x0000000100 [0x410fd082]
[  163.049645] CPU4 is up
[  163.050492] Detected PIPT I-cache on CPU5
[  163.050505] GICv3: CPU5: found redistributor 101 region 0:0x0000000051ba0000
[  163.050527] CPU5: Booted secondary processor 0x0000000101 [0x410fd082]
[  163.051134] CPU5 is up
[  163.051944] CPU3: update max cpu_capacity 363
[  163.051950] CPU4: update max cpu_capacity 1024
[  163.077256] PM: dpm_run_callback(): genpd_resume_noirq+0x0/0x4 returns -16
[  163.077262] PM: Device hdmi_pixel_link_clk failed to resume noirq: error -16
[  163.122498] [drm] Started firmware!
[  163.123633] [drm] HDP FW Version - ver 34219 verlib 20560
[  163.125013] [drm] Mode: 1920x1080p148500
[  163.151966] [drm] Pixel clock: 148500 KHz, character clock: 148500, bpc is 8-bit.
[  163.151969] [drm] VCO frequency is 2970000 KHz
[  163.216401] [drm] Sink Not Support SCDC
[  163.217926] [drm] No vendor infoframe
[  163.317315] ahci-imx 5f020000.sata: external osc is used.
[  163.325274] PM: resume devices took 0.200 seconds
[  163.540871] OOM killer enabled.
[  163.544016] Restarting tasks ... done.
[  163.555059] Resume cause unknown
[  163.558554] PM: suspend exit
[  163.634698] ata1: SATA link down (SStatus 0 SControl 300)
[  163.671072] misc FreescaleGyroscope: mma enable setting active

[Zhiming_Liu]

Hi 

I need reroduce this issue firstly and give your feedback

[Zhiming_Liu]

Can you attach your steps?

[RKhairulin]

I've edited my question, please read again.