LS2088A: Strange crashes with U-boot binary on custom board

[rams]

We are facing a strange issue with U-boot on LS2088A custom hardware that makes the card crash during boot. 

When we make a code change in U-boot and make a new U-boot binary, we see crash during kernel boot 
or u-boot startup. 
If we alter the U-boot binary size by increasing or decreasing the binary size by adding spaces in the string, 
the problem goes away.  

We have tested DDR and we have stable DDR parameters from DDRV tool.  We ran memory tests and we do not have
any memory issues. 
We do not see any DDR failures.  We have tested on several custom boards, the issue can be reproduced on 
many Ls2088a custom hardware. 
We also have ls1046 and ls1043a hardware and we have not seen such issue on these type of hardware.

If we manage to build the binary and successfully run on the board, we do not see any issues with the 
binary on any other card. 
We have to work around this issue by adding extra spaces in the board specific file /ls2080a.c 



-    printf("\n" BOARD_NAME " MMI DIP-Swich value: 0x%x\n               ", dip_switch); 
+    printf("\n" BOARD_NAME " MMI DIP-Swich value: 0x%x\n                   ", dip_switch);  



The only change we made to workaround the crash was to add 4 spaces in the printf. 

Please see the attached log for the crash.

Your feedback will be very useful to investigate this issue further.  
Do you have any clue on this issue based on our obervations?

Thanks
Ram

[yipingwang]

Hello Ramasubramanian Subramanian,

Which version LSDK are you using now?

I checked the file board/freescale/ls2080a/ls2080a.c in LSDK 1809, 1806 and 1803, there is no the following sentence at all.

printf("\n" BOARD_NAME " MMI DIP-Swich value: 0x%x\n               ", dip_switch); 

Have a great day,
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[rams]

Hi,

We are using LSDK 1809.  We have tested our custom hardware on 1803, 1806 and 1809.  This is our custom board file for the custom hardware based on LS2088A.  

Do you infer anything from the syptom of memory corruption? How can we go about debugging this issue?

Thanks

Ram

[yipingwang]

Hello Ramasubramanian Subramanian,

In your log, it seems that it failed at the begging of Kernel starting up.

Please refer to the section "3. Debug Linux Kernel by Attaching Running U-boot" in

the document Use CodeWarrior for ARMv8 to Debug U-boot and Linux Kernel and Bring up Bare Boards to attach to u-boot and set the breakpoint at the beginning of Linux Kernel to debug Linux Kernel.

In addition, please deploy PPA firmware on the NOR flash under u-boot.

Have a great day,
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[rams]

Hi,

We have deployed PPA firmware in the NOR flash.  I will attach the console logs that shows PPA inititalization as well and still having crash.  This crash is not always on linux start but can be even early as well.

U-Boot 2018.03-00001-gfee6f768e9 (Nov 01 2018 - 13:31:00 +0100)

SoC: LS2088AE Rev1.0 (0x87090010)

Clock Configuration:

CPU0(A72):1600 MHz CPU1(A72):1600 MHz CPU2(A72):1600 MHz

CPU3(A72):1600 MHz CPU4(A72):1600 MHz CPU5(A72):1600 MHz

CPU6(A72):1600 MHz CPU7(A72):1600 MHz

Bus: 600 MHz DDR: 1600 MT/s DP-DDR: 1600 MT/s

Reset Configuration Word (RCW):

00000000: 40303030 40400040 00000000 00000000
00000010: 00000000 00000000 00200000 00000000
00000020: 01001180 00002580 00000000 00000000
00000030: 007fe90b 00000000 00000000 00000000
00000040: 00000000 00000000 00000000 00000000
00000050: 00000000 00000000 00000000 00000000
00000060: 00000000 00000000 00027000 00000000
00000070: 3f28000c 00000002
Model: 2080a custom Board

Board: LS2088AE Rev1.0-10G-N2SW, SERDES1 Reference : Clock1 = 156.25MHz Clock2 = 125MHz
SERDES2 Reference : Clock1 = 100MHz Clock2 = 100MHz
I2C: ready
DRAM: Initializing DDR....SDRAM
DP-DDR: 4.9 GiB
DDR 3.9 GiB (DDR4, 64-bit, CL=15, ECC on)
DDR Controller Interleaving Mode: 256B
DP-DDR 1 GiB (DDR4, 32-bit, CL=15, ECC on)
Device tree error at node 'interrupt-controller@6005▒'
Some drivers failed to bind
initcall sequence 00000000fbdc2108 failed at call 00000000301197a0 (err=-11)
### ERROR ### Please RESET the board ###

I few  questions regarding PPA.

1) We have moved the PPA firmware in the flash 0x580400000 to 0x580c00000 and updated the CONFIG_SYS_LS_PPA_FW_ADDR to 0x580c00000  for our cutom configuration.  Is that fine to keep PPA firmware on any NOR flash location as long as we keep CONFIG_SYS_LS_PPA_FW_ADDR updated to PPA firmware address. on NOR flash.

2)  We compile PPA from the source at:

qoriq-components/ppa-generic - The Primary Protected Application 

We use PPA binary from platform-rdb for our custom board.  Do we have to create platform specific files like ddr_init.c for our custom board and make a cutom ppa binary for custom board?

Please clarify

Thanks

Ram