T1022: uboot gets struck after reboot

[hemwant]

We are having a custom board having T1022 processor with Apacer DDR4 4GB ECC DDR4 2400 CL17 (78.B1GMV.4030B) , the card boots up properly on power sequencing but after few reboots from user space , the card gets struck at uboot. The log of the same is attached below.

[platform: libdwdm] i2c_device_write: <Debug> i2c_bus 0, mux_addr 0x0, channel_addr 0x0, dev_addr 0x5c, offset 0x3e, size 1
[platform: libdwdm] i2c_device_write: <Debug> Write to I2C device failed, errno 6, bytes written -1
[platform: libdwdm] i2c_device_write: <Debug> i2c_bus 0, mux_addr 0x0, channel_addr 0x0, dev_addr 0x5c, offset 0x42, size 1
[platform: libdwdm] i2c_device_write: <Debug> Write to I2C device failed, errno 6, bytes written -1
Sep  8 15:11:22 t1042d4rdb local5.err TPN[1493]: setLedColor: Error in dwdm_trnscvr_led_ctrl 13 interface 3
Stopping OpenBSD Secure Shell server: sshdSep  8 15:11:23 t1042d4rdb auth.info sshd[1363]: Received signal 15; terminating.
stopped /usr/sbin/sshd (pid 1363)
.
hwclock: can't open '/dev/misc/rtc': No such file or directory
Stopping network benchmark server: netserverstopped /usr/sbin/netserver (pid 1386)
.
Stopping system log daemon...Sep  8 15:11:23 t1042d4rdb syslog.info syslogd exiting
0
Stopping kernel log daemon...1
Stopping internet superserver: xinetd.
Deconfiguring network interfaces... done.
Sending all processes the TERM signal...
Sep  8 15:11:23 t1042d4rdb syslog.info syslogd exiting
Sending all processes the KILL signal...
Unmounting remote filesystems...
Deactivating swap...
Unmounting local filesystems...
UBIFS (ubi0:0): background thread "ubifs_bgt0_0" stops
Rebooting... reboot: Restarting system


Initializing....using SPD
Not enough bank(chip-select) for CS0+CS1 on controller 0, interleaving disabled!
2 GiB left unmapped
Loading second stage boot loader .................................................................................................Ba0
**bleep**: 8000050C XER: 00000000 LR: FFFD8B6C REGS: fffd7ed0 TRAP: 0e00 DAR: 00000000
MSR: 00000000 EE: 0 PR: 0 FP: 0 ME: 0 IR/DR: 00

GPR00: FFFE294C FFFD7FC0 FFFC8000 300BFFFF 000BFFFF FFFCB140 FFFCB140 FFFCB130
GPR08: 00000000 30000000 30000000 FFFD7FC0 FFFE21D8 1001ACD4 00000000 00000000
GPR16: 00000000 00000001 00000000 00000001 00000000 00000000 00000000 8000050C
GPR24: 00000000 10010000 10012CF0 00000000 00102000 FFFCB0F0 FFFE7290 00102000
Call backtrace:
FFFE294C FFFDB1B0 FFFD8E84 BEEFDEAD

[bushra]

@Bulat 

We have tried debugging this issue using Code warrior Tap tool. While creating QorIq Configuration Project, read SPD option shows error. 

Even if I create an autoconfiguration file, with DIMM data from datasheet, in DDR validation tab, target connection cannot be made.

This is a custom board in which we have previously used lauterbach tool as well so I dont think there should be an issue in the jtag chain detection.

Processor is T1022 rev 1.1

Snapshots attached. Please advise.

[Bulat]

It is difficult to guess what is going on, probably schematics issue. Can you re-produse this problem on NXP board, like T1022RDB?

Regards,

Bulat

[hemwant]

@Bulat  We have assembled around 10 cards and this issue is present in only 2 cards, probably this is not schematic related issue. 

We cannot produce this issue on NXP board  [T104xRDB].

Could you suggest something to identify the root cause in this case?

[Bulat]

As I wrote, it is very difficult to gess why boot fails when previous boot was successful. Especially if other same boards have no such problems at all.

Check reset circuit going to DDR4 devices. Normally we recommend that DDR reset is generated from processor's HRESET signal.

Try to compare boot logs of both, successful and failing boots, probably this can hint something.

[kashishanand]

A few updates

1. DDR reset is being generated from processor's HRESET,still issue persists.

2. We have enabled a few debug prints and shared boot logs of both cases; successful and failing boots.

The file with the debug changes has been attached for your reference.

uboot gets stuck in second stage bootloader, this behavior is intermittent.