ls1046 freeze for 2 minutes after reboot (imx2-wdt 2ad0000.watchdog: Device shutdown: Expect reboot)

[pankaj17087]

Hi, 

We are using nxp linux-5.15.71  using from meta-qoriq repo(https://github.com/nxp-qoriq/meta-qoriq/blob/kirkstone/recipes-kernel/linux/linux-qoriq_5.15.bb) kirkstone branch. We are running linux-5.15 kernel image on LS1046AE SOC. When we were doing a reboot, the system got stuck for 120sec  after this message ("Rebooting... [ 219.689967] imx2-wdt 2ad0000.watchdog: Device shutdown: Expect reboot!" ) and then firmware and u-boot start booting. We are using "U-Boot 2019.04+fsl+gce862bb"

 We are not observing the same reboot freezing issue for NXP 5.10.52 kernel version(https://github.com/Freescale/meta-freescale/blob/master/recipes-kernel/linux/linux-qoriq_5.10.bb).  For reference please find the linux-5.15 reboot log below:

Stopping logproxyd: done
/bin/tar: Removing leading `/' from member names
/bin/tar: Removing leading `/' from member names
Stopping RASdaemon services: done.
Stopping schema-init: done
stopping rsyslogd ... done
Stopping NVPD: done
Sending all processes the TERM signal...
Sending all processes the KILL signal...
Deactivating swap...
Unmounting local filesystems...
/sbin/umount.aufs:au_nftw.c:76: /var/lib/dhcp/dhcpd.leases~: Read-only file system
/sbin/umount.aufs:au_nftw.c:76: /var/lib/dhcp/dhcpd.leases~: Read-only file system
umount: /mnt/rw: target is busy.
[ 219.663008] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null). Quota mode: disabled.
/etc/init.d/rc: line 48: .error_null: Read-only file system
Rebooting... [ 219.689967] imx2-wdt 2ad0000.watchdog: Device shutdown: Expect reboot![ 219.697650] reboot: Restarting system
NOTICE: UDIMM 18ASF2G72HZ-3G2R1
Warm reset: clear D_INIT
Warm reset: set SDRAM_CFG_BI

---> Freeze here, taking 2 min to restart the system

Please let us know any pointers to fix this issue. 

Regards,

Pankaj Pandey

[June_Lu]

So you are using the customer board instead of LS1046ARDB which is NXP reference board, is it correct?

If your customer board could not boot every time, could help explain how do you get the 5.15 log?

It seems you login the 5.15.

Thanks

[June_Lu]

From the issue, there is "Warm reset"; but in the log there is "Cold reset", could help explain these differences?

Thanks

[pankaj17087]

warm reset is the correct one. it is coming after reboot of linux-5.15 or linux-5.10. Please ignore the cold reset one because it is coming after the power cycle of the board.

[June_Lu]

From the log 5.15 attached:

From line 1-3

NOTICE:  UDIMM 18ASF2G72HZ-3G2R1

Cold reset: set D_INIT

Cold reset: clear SDRAM_CFG_BI

But issue pasted:

NOTICE: UDIMM 18ASF2G72HZ-3G2R1

Warm reset: clear D_INIT

Warm reset: set SDRAM_CFG_BI

Which will be the correct one?

Thanks

[pankaj17087]

Thanks for the update June_li.  We are using customer hardware with some local changes in the existing atf-1.5 version. So our limitation is not to upgrade to atf-2.6 in existing hardware.  Can you please suggest any pointers that can be checked in Linux-5.15 compared to linux-5.10?   

Freezing issue observed during ATF-1.5 booting:- 

NOTICE: UDIMM 18ASF2G72HZ-3G2R1

Cold reset: set D_INIT

Cold reset: clear SDRAM_CFG_BI

<2 minutes freezing>

[June_Lu]

Please follow link below to get default image for kernel 5.15.71:

Layerscape Linux Distribution POC User Guide, Rev. L5.15.71-2.2.0 , 4.5 Build Layerscape LDP with Yocto bitbake

There is no issue with default image for kernel 5.15.71

[kenli]

You can try https://www.nxp.com/design/design-center/software/embedded-software/linux-software-and-development-t...
 or LSDK2108 as bsp.

[June_Lu]

Could you attach the full logs respectively Kernel-Version: 5.15 or 5.10

[pankaj17087]

Hi, 

Please find 5.10 and 5.15 bootup log as an attachment. 

Regards,

Pankaj 

[June_Lu]

Please check below information:

What's the rootfs type you used? yocto or ubuntu or others?

Had you tried full image of LF release for kernel 5.15.71?

Had you added some application built from kernel code?

[pankaj17087]

Hi, Please find the inline detail below ;

What's the rootfs type you used? yocto or ubuntu or others?

Pankaj:  Using same yocto-4.0(krickstone) base rootfs booth nxp-5.10 and nxp-5.15. but issue observed in 5.15.

Had you tried full image of LF release for kernel 5.15.71?

Pankaj: what is LF? we are using full image built from our yocto-4.0 image for both 5.10 and 5.15. 

Had you added some applications built from kernel code?

Pankaj: We are booting at run level 1 where avoiding running any added application.

We also observed the below-mentioned patch, introduced in the 5.17 kernel which mentioned about 2 min freeze, please check if it is relevant to the issue that we are observing in 5.15 for ls1046ardb

https://github.com/torvalds/linux/commit/cf6e4bc16dba2068fa4a115300185a8c56edc93f

[June_Lu]

Could you share if it's LS1046ARDB or customer board?

[pankaj17087]

yes, It is LS1046ARDB board.

Kernel-Version: 5.15 or 5.10
CPU: LS1046ARDB(qoriq family)