Running into an issue when a "reboot" in Linux is happened. From time to time (1 out of 20) this ends up in a strange situation. Linux shut down properly but Kernel doesn't boot up completly.
I am a bit clueless how to find the reason. My findings so far:
* a powercycle or an reset to PMIC (via I2C): and all is fine again.
* Depending on kernel config and kernel version the boot stops at different stages: from only "Starting Kernel" til full boot anything can happen (but with same version it is reproducable, once we are in this situation).
* Depending on kernel and config it can happen that watchdog is triggered - than we end up in a bootloop. Or it keeps into kernel printing:
[ 22.283106] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 22.289202] rcu: 1-...0: (1 ticks this GP) idle=645/1/0x4000000000000000 softirq=45/45 fqs=2626
[ 22.298082] (detected by 2, t=5254 jiffies, g=-1127, q=13)
[ 22.303657] Task dump for CPU 1:
[ 22.306885] task:swapper/0 state:R running task stack: 0 pid: 1 ppid: 0 flags:0x0000000a
[ 22.316808] Call trace:
[ 22.319251] __switch_to+0x108/0x160
[ 22.322838] ___slab_alloc+0x654/0x7b4
[ 22.326595] __slab_alloc.constprop.0+0x38/0x7c
[ 22.331130] 0xffff0000034a0000
[ 31.149558] imx2_wdt_ping
(watchdog is still serviced, so it doesn't stop)
* When I use a kernel that boots up completly - this doesn't "heal" the situation. Booting after that with our standard kernel it hangs again.
* clk dump in uboot is same (whether SoC is in this situation or normal)
* Looking at NXP-EVK board any kind of reboot (linux) or restart (uboot) is stated as POR in uboot? That sounds strange to me. My board is always show WDOG, except on a power cycle (POR) - which seems correct to me.
I am wondering what can be the reason? Clocks seems not to be root cause.. It could be some misconfigured registers, memory or power domains. Is the reset of SoC is done properly?
Is in Kernel anything board specific what could go wrong or is missed? Where is the final reset command in Linux kernel or does reboot triggers the WDOG to finally make the reset?
I am thinking of a workaround to include in Uboot this restart of PMIC via I2C, but it has to run only if no POR is recognised. Otherwise it board will never come up. Another way would be to included that PMIC reset in shutdown of kernel. But these are all dirty workarounds....
Any ideas? (Our board uses different PMIC than NXP, tried different uboot up to 2024.07 and kernels).
