Random shutdowns due to watchdogs

bgarcia — Wed, 24 Jan 2024 10:17:47 GMT

I am using BSP37.0 and I can observe random reboots in the Linux instance.

[␛[0;32m OK ␛[0m] Started ␛[0;1;39mSerial Getty on ttyLF0␛[0m. [␛[0;32m OK ␛[0m] Reached target ␛[0;1;39mLogin Prompts␛[0m. [␛[0;32m OK ␛[0m] Reached target ␛[0;1;39mMulti-User System␛[0m. Starting ␛[0;1;39mRecord Runlevel Change in UTMP␛[0m... [␛[0;32m OK ␛[0m] Finished ␛[0;1;39mRecord Runlevel Change in UTMP␛[0m. Auto Linux BSP 37.0 s32g274ardb2 ttyLF0 s32g274ardb2 login: Stopping ␛[0;1;39mSession c4 of User root␛[0m... [␛[0;32m OK ␛[0m] Removed slice ␛[0;1;39mSlice /system/modprobe␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mMulti-User System␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mLogin Prompts␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mRPC Port Mapper␛[0m. [␛[0;32m OK Stopping ␛[0;1;39mSerial Getty on ttyLF0␛[0m... Stopping ␛[0;1;39mOpenSSH Per-Conne…Daemon (192.168.0.96:45432)␛[0m... Stopping ␛[0;1;39mSystem Logging Service␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mResets System Activity Logs␛[0m. Stopping ␛[0;1;39mLoad/Save Random Seed␛[0m... Stopping ␛[0;1;39mXinetd A Powerful Replacement For Inetd␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mJob spooling tools␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mPeriodic Command Scheduler␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mirqbalance daemon␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mSystem level shared mutex initializer␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mHardware RNG Entropy Gatherer Daemon␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mSystem Logging Service␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mNetperf Benchmark Server␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mNFS status monitor for NFSv2/3 locking.␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mRespond to IPv6 Node Information Queries␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mNetwork Router Discovery Daemon␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mXinetd A Powerful Replacement For Inetd␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mGetty on tty1␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mSerial Getty on ttyLF0␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mOpenSSH Per-Connec…n Daemon (192.168.0.96:45432)␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mLoad/Save Random Seed␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mSession c4 of User root␛[0m. [␛[0;32m OK ␛[0m] Removed slice ␛[0;1;39mSlice /system/getty␛[0m. [␛[0;32m OK ␛[0m] Removed slice ␛[0;1;39mSlice /system/serial-getty␛[0m. [␛[0;32m OK ␛[0m] Removed slice ␛[0;1;39mSlice /system/sshd␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mHost and Network Name Lookups␛[0m. Stopping ␛[0;1;39mUser Login Management␛[0m... Stopping ␛[0;1;39mUser Manager for UID 0␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mOpenSSH Key Generation␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mUser Manager for UID 0␛[0m. Stopping ␛[0;1;39mUser Runtime Directory /run/user/0␛[0m... [␛[0;32m OK ␛[0m] Unmounted ␛[0;1;39m/run/user/0␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mUser Runtime Directory /run/user/0␛[0m. [␛[0;32m OK ␛[0m] Removed slice ␛[0;1;39mUser Slice of UID 0␛[0m. Stopping ␛[0;1;39mD-Bus System Message Bus␛[0m... Stopping ␛[0;1;39mPermit User Sessions␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mD-Bus System Message Bus␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mUser Login Management␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mPermit User Sessions␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mNetwork␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mRemote File Systems␛[0m. Stopping ␛[0;1;39mNetwork Name Resolution␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mNetwork Name Resolution␛[0m. Stopping ␛[0;1;39mNetwork Configuration␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mNetwork Configuration␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mPreparation for Network␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mIPv6 Packet Filtering Framework␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mIPv4 Packet Filtering Framework␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mBasic System␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mPath Units␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mDispatch Password …ts to Console Directory Watch␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mForward Password R…uests to Wall Directory Watch␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mSlice Units␛[0m. [␛[0;32m OK ␛[0m] Removed slice ␛[0;1;39mUser and Session Slice␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mSocket Units␛[0m. [␛[0;32m OK ␛[0m] Closed ␛[0;1;39mD-Bus System Message Bus Socket␛[0m. [␛[0;32m OK ␛[0m] Closed ␛[0;1;39msshd.socket␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mSystem Initialization␛[0m. [␛[0;32m OK ␛[0m] Closed ␛[0;1;39mSyslog Socket␛[0m. [␛[0;32m OK ␛[0m] Closed ␛[0;1;39mNetwork Service Netlink Socket␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mApply Kernel Variables␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mLoad Kernel Modules␛[0m. Stopping ␛[0;1;39mNetwork Time Synchronization␛[0m... [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mNetwork Time Synchronization␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mCreate Volatile Files and Directories␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mLocal File Systems␛[0m. Unmounting ␛[0;1;39m/tmp␛[0m... Unmounting ␛[0;1;39m/var/volatile/tmp␛[0m... [␛[0;32m OK ␛[0m] Unmounted ␛[0;1;39m/tmp␛[0m. [␛[0;32m OK ␛[0m] Unmounted ␛[0;1;39m/var/volatile/tmp␛[0m. Unmounting ␛[0;1;39m/var/volatile␛[0m... [␛[0;32m OK ␛[0m] Unmounted ␛[0;1;39m/var/volatile␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mPreparation for Local File Systems␛[0m. [␛[0;32m OK ␛[0m] Stopped target ␛[0;1;39mSwaps␛[0m. [␛[0;32m OK ␛[0m] Reached target ␛[0;1;39mUnmount All Filesystems␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mRemount Root and Kernel File Systems␛[0m. [␛[0;32m OK ␛[0m] Stopped ␛[0;1;39mCreate Static Device Nodes in /dev␛[0m. [␛[0;32m OK ␛[0m] Reached target ␛[0;1;39mSystem Shutdown␛[0m. [␛[0;32m OK ␛[0m] Reached target ␛[0;1;39mLate Shutdown Services␛[0m. [␛[0;32m OK ␛[0m] Finished ␛[0;1;39mSystem Reboot␛[0m. [␛[0;32m OK ␛[0m] Reached target ␛[0;1;39mSystem Reboot␛[0m. [44710.450861] systemd-shutdown[1]: Using hardware watchdog 's32cc watchdog', version 0, device /dev/watchdog0 [44710.450900] systemd-shutdown[1]: Failed to set timeout to 10min: Invalid argument [44710.479157] systemd-shutdown[1]: Syncing filesystems and block devices. [44710.495131] systemd-shutdown[1]: Sending SIGTERM to remaining processes... [44710.508589] systemd-journald[123]: Received SIGTERM from PID 1 (systemd-shutdow). [44710.527190] systemd-shutdown[1]: Sending SIGKILL to remaining processes... [44710.537177] systemd-shutdown[1]: Unmounting file systems. [44710.539229] [983]: Remounting '/' read-only with options 'n/a'. [44710.548239] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null). Quota mode: disabled. [44710.557310] systemd-shutdown[1]: All filesystems unmounted. [44710.557327] systemd-shutdown[1]: Deactivating swaps. [44710.557415] systemd-shutdown[1]: All swaps deactivated. [44710.557425] systemd-shutdown[1]: Detaching loop devices. [44710.557828] systemd-shutdown[1]: All loop devices detached. [44710.557838] systemd-shutdown[1]: Stopping MD devices. [44710.558070] systemd-shutdown[1]: All MD devices stopped. [44710.558083] systemd-shutdown[1]: Detaching DM devices. [44710.558290] systemd-shutdown[1]: All DM devices detached. [44710.558301] systemd-shutdown[1]: All filesystems, swaps, loop devices, MD devices and DM devices detached. [44710.631657] systemd-shutdown[1]: Syncing filesystems and block devices. [44710.632795] systemd-shutdown[1]: Rebooting. [44710.638199] pfeng 46000000.pfe: Suspending driver [44710.638578] pfeng 46000000.pfe pfe0: Link is Down [44710.639183] pfeng 46000000.pfe pfe1: Link is Down [44710.652541] pfeng 46000000.pfe: HIF3 not enabled, skipped [44710.652553] pfeng 46000000.pfe: HIF2 stopped [44710.683929] pfeng 46000000.pfe: HIF2 disabled [44710.683942] pfeng 46000000.pfe: HIF1 stopped [44710.707884] pfeng 46000000.pfe: HIF1 disabled [44710.707895] pfeng 46000000.pfe: HIF0 stopped [44710.727888] pfeng 46000000.pfe: HIF0 disabled [44710.743690] pfeng 46000000.pfe: Global poller finished [44710.743769] pfeng 46000000.pfe: Removing default logical interface (pfe0, parent: emac0) [44710.743832] pfeng 46000000.pfe: pfe0 (p0x0000000018e0081e) removed from emac0 (p0x00000000a9506e0f) [44710.743994] pfeng 46000000.pfe: Removing default logical interface (hif0-logif, parent: hif0) [44710.744053] pfeng 46000000.pfe: hif0-logif (p0x000000000c7d2183) removed from hif0 (p0x000000002a3af9db) [44710.744211] pfeng 46000000.pfe: Removing default logical interface (pfe1, parent: emac1) [44710.744270] pfeng 46000000.pfe: pfe1 (p0x00000000b45034b6) removed from emac1 (p0x00000000cec7b85d) [44710.744426] pfeng 46000000.pfe: Removing default logical interface (hif1-logif, parent: hif1) [44710.744483] pfeng 46000000.pfe: hif1-logif (p0x000000005584197e) removed from hif1 (p0x0000000065459d60) [44710.744642] pfeng 46000000.pfe: Removing default logical interface (pfe2, parent: emac2) [44710.744699] pfeng 46000000.pfe: pfe2 (p0x0000000075ebd0eb) removed from emac2 (p0x000000002b20084b) [44710.744855] pfeng 46000000.pfe: Removing default logical interface (hif2-logif, parent: hif2) [44710.744914] pfeng 46000000.pfe: hif2-logif (p0x000000008aac08b9) removed from hif2 (p0x000000001f3b440d) [44711.452216] pfeng 46000000.pfe: Stopping rtable worker... [44711.452311] pfeng 46000000.pfe: rtable worker stopped [44711.452804] pfeng 46000000.pfe: PFE Platform stopped [44711.452907] pfeng 46000000.pfe: PFE controller off [44711.453184] ci_hdrc ci_hdrc.0: remove, state 4 [44711.453199] usb usb1: USB disconnect, device number 1 [44711.453890] ci_hdrc ci_hdrc.0: USB bus 1 deregistered [44711.563946] sja1110 spi5.0: [sja1110_remove] [44711.564123] sja1110 spi5.1: [sja1110_remove] [44711.564925] s32cc-wdt 4010c000.watchdog: Device shutdown: Expect reboot! [44711.565047] reboot: Restarting system [44711.565053] printk: enabled sync mode [44711.567669] printk: console [ttyLF0]: printing thread stopped NOTICE: Reset status: Destructive Reset (RUN) NOTICE: BL2: v2.5(release):bsp37.0-2.5 NOTICE: BL2: Built : 09:12:21, Jun 13 2023 NOTICE: BL2: Booting BL31 U-Boot 2020.04+g8f2fa84f9e (Jan 16 2024 - 15:34:46 +0000)

As you can see, this happened at boot time 44710. Looks like the watchdog not being able to be feeded is the root cause

[44710.450861] systemd-shutdown[1]: Using hardware watchdog 's32cc watchdog', version 0, device /dev/watchdog0 [44710.450900] systemd-shutdown[1]: Failed to set timeout to 10min: Invalid argument

Are you aware of this?

Re: Random shutdowns due to watchdogs

Daniel-Aguirre — Thu, 25 Jan 2024 17:58:38 GMT

Hi,

Are you using any NXP reference boards? Or is this a custom board?

Was this BSP built by you? Or did you take the pre-built BSP provided by NXP (under a *.tgz file)?

From our side, we don't see the behavior you are seeing, we can successfully boot our RDB2 with BSP37.0.

Please, let us know.

topic Random shutdowns due to watchdogs in S32G

Random shutdowns due to watchdogs

Re: Random shutdowns due to watchdogs