What is the next steps to debug if DDR training failed ?

[Jimmychea]

Hi, i got an error when booting on my custom board which is "DDR training failed"(Uboot). May i know how to further debug to identify the DDR root cause ?

Note: my custom board is following the i.MX8M Plus EVK design, using same type and volume of the LPDDR4 RAM.

[joanxie]

did you pass the DDR stress test? pls check the link as below:

"https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX-8M-Family-DDR-Tool-Release/ta-p/110..."

[Jimmychea]

When i click "Callibration" on DDR tool, the result showing failed as below,

Download is complete
Waiting for the target board boot...

===================hardware_init=====================

********Found PMIC PCA9450**********
hardware_init exit

*************************************************************************

*************************************************************************

*************************************************************************
MX8 DDR Stress Test V3.30
Built on Nov 24 2021 13:52:12
*************************************************************************

Waiting for board configuration from PC-end...

--Set up the MMU and enable I and D cache--
- This is the Cortex-A53 core
- Check if I cache is enabled
- Enabling I cache since it was disabled
- Push base address of TTB to TTBR0_EL3
- Config TCR_EL3
- Config MAIR_EL3
- Enable MMU
- Data Cache has been enabled
- Check system memory register, only for debug

- VMCR Check:
- ttbr0_el3: 0x97d000
- tcr_el3: 0x2051c
- mair_el3: 0x774400
- sctlr_el3: 0xc01815
- id_aa64mmfr0_el1: 0x1122

- MMU and cache setup complete

*************************************************************************
ARM clock(CA53) rate: 1800MHz
DDR Clock: 2000MHz

============================================
DDR configuration
DDR type is LPDDR4
Data width: 32, bank num: 8
Row size: 17, col size: 10
Two chip selects are used
Number of DDR controllers used on the SoC: 1
Density per chip select: 3072MB
Density per controller is: 6144MB
Total density detected on the board is: 6144MB
============================================

MX8M-plus: Cortex-A53 is found

*************************************************************************

============ Step 1: DDRPHY Training... ============
---DDR 1D-Training @2000Mhz...
PMU: Error: CA Training Failed.
PMU: ***** Assertion Error - terminating *****
[Result] FAILED

============ Step 1: DDRPHY Training... ============
---DDR 1D-Training @2000Mhz...
PMU: Error: CA Training Failed.
PMU: ***** Assertion Error - terminating *****
[Result] FAILED

[joanxie]

you can refer to the link as below:

"https://community.nxp.com/t5/i-MX-Processors/i-MX8M-Nano-LPDDR4-Calibration-failed-CA-Training-Faile..."

[Jimmychea]

What are the steps i can try/do to find out the root cause ? Because some of our custom board's LPDDR4 is working, but some experiences DDR training failed.

[bernhardfink]

Try it with a lower clock frequency.

Your setting:

ARM clock(CA53) rate: 1800MHz
DDR Clock: 2000MHz

This seems to be pretty high, maybe too high.

The Cortex-A53 frequency can be reduced in the GUI: 1800/1600/1200

In the RPA Excel sheet, if I set 1600MHz in row #29, the DDR clock results just in a TBD note in the .ds file.
You can look into the .ds file for DDR4 for a 1600MHz setting or into the .ds file for DDR3L for 933MHz

memory set 0x30360054 32 0x190032 #DRAM_PLL_FDIV_CTL0: For 1600MHz, pll_main_div = 400, pll_pre_div = 3, pll_post_div = 2

memory set 0x30360054 32 0x137023 #DRAM_PLL_FDIV_CTL0: For 933MHz, pll_main_div = 311, pll_pre_div = 2, pll_post_div = 3

ARM clock(CA53) rate: 1200MHz
DDR Clock: 1600MHz

You can go lower for test purposes, if it still doesn't pass the training, then something is really wrong.

Regards,
Bernhard.

[Jimmychea]

is it necessary to reduce A53 frequency ? Will A53 frequency impact DDR  ?

I have been tested to reduce the DDR frequency to 800MHz and A53 frequency remained 1800MHz, but it still got DDR training failed.

[bernhardfink]

In principle it is not necessary to reduce the frequency, but if the test highest allowed frequency fails, the it's useful to try it with reduced frequency.

If you have other boards which are working, then your schematic design is ok and the PCB layout is also not so bad in the first step. If the PCB design would be somewhere on a performance limit, you would bring the failing boards back to work with a test at reduced DDR frequency. But as this is not the case, my guess is that you have an assembly problem and some signals are not connected correctly. For the training phase quite some signals from the CA bus are involved, if one of these signals is not connected or has a shortcut, then the training fails.

In the .ds file you find an option to skip the training (close to the end of the skript), maybe you can get into a memory write/read test phase with that. Getting results from a write/read test could tell us which signal(s) may have an issue.

Regards,
Bernhard.

[Jimmychea]

In order to skip the CA training, change the value from "0x131f" to "0x031f" ?

[Jimmychea]

Detailed Training Log

Download is complete
Waiting for the target board boot...

===================hardware_init=====================

********Found PMIC PCA9450**********
hardware_init exit

*************************************************************************

*************************************************************************

*************************************************************************
MX8 DDR Stress Test V3.30
Built on Nov 24 2021 13:52:12
*************************************************************************

Waiting for board configuration from PC-end...

--Set up the MMU and enable I and D cache--
- This is the Cortex-A53 core
- Check if I cache is enabled
- Enabling I cache since it was disabled
- Push base address of TTB to TTBR0_EL3
- Config TCR_EL3
- Config MAIR_EL3
- Enable MMU
- Data Cache has been enabled
- Check system memory register, only for debug

- VMCR Check:
- ttbr0_el3: 0x97d000
- tcr_el3: 0x2051c
- mair_el3: 0x774400
- sctlr_el3: 0xc01815
- id_aa64mmfr0_el1: 0x1122

- MMU and cache setup complete

*************************************************************************
ARM clock(CA53) rate: 1800MHz
DDR Clock: 2000MHz

============================================
DDR configuration
DDR type is LPDDR4
Data width: 32, bank num: 8
Row size: 17, col size: 10
Two chip selects are used
Number of DDR controllers used on the SoC: 1
Density per chip select: 3072MB
Density per controller is: 6144MB
Total density detected on the board is: 6144MB
============================================

MX8M-plus: Cortex-A53 is found

*************************************************************************

============ Step 1: DDRPHY Training... ============
---DDR 1D-Training @2000Mhz...
PMU10: **** Start LPDDR4 Training. PMU Firmware Revision 0x1007 ****
PMU10: Setting boot clock divider to 40
PMU10: PHY TOTALS - NUM_DBYTES 4 NUM_NIBBLES 8 NUM_ANIBS 10
PMU10: CSA=0x03, CSB=0x03, TSTAGES=0x131F, HDTOUT=5, MMISC=0 DRAMFreq=4000MT DramType=LPDDR4
PMU10: Pstate0 MRS MR01_A0=0xF4 MR02_A0=0x3F MR03_A0=0x33 MR11_A0=0x66
PMU10: Pstate0 MRS MR12_A0=0x48 MR13_A0=0x00 MR14_A0=0x48 MR22_A0=0x16
PMU10: Pstate0 MRS MR01_A1=0xF4 MR02_A1=0x3F MR03_A1=0x33 MR11_A1=0x66
PMU10: Pstate0 MRS MR12_A1=0x48 MR13_A1=0x00 MR14_A1=0x48 MR22_A1=0x16
PMU10: Pstate0 MRS MR01_B0=0xF4 MR02_B0=0x3F MR03_B0=0x33 MR11_B0=0x66
PMU10: Pstate0 MRS MR12_B0=0x48 MR13_B0=0x00 MR14_B0=0x48 MR22_B0=0x16
PMU10: Pstate0 MRS MR01_B1=0xF4 MR02_B1=0x3F MR03_B1=0x33 MR11_B1=0x66
PMU10: Pstate0 MRS MR12_B1=0x48 MR13_B1=0x00 MR14_B1=0x48 MR22_B1=0x16
PMU5: CA bitmap dump for cs 0
PMU5: CAA0 fffffff000000000000fffffffffffffffffffffffffffff
PMU5: CAA1 fffffff0000000000001ffffffffffffffffffffffffffff
PMU5: CAA2 fffffff8000000000000ffffffffffffffffffffffffffff
PMU5: CAA3 ffffffe0000000000007ffffffffffffffffffffffffffff
PMU5: CAA4 ffffffff0000000000001fffffffffffffffffffffffffff
PMU5: CAA5 fffffff8000000000003ffffffffffffffffffffffffffff
PMU5: CA bitmap dump for cs 1
PMU5: CAA0 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAA1 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAA2 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAA3 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAA4 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAA5 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CA bitmap dump for cs 0
PMU5: CAB0 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB1 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB2 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB3 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB4 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB5 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CA bitmap dump for cs 1
PMU5: CAB0 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB1 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB2 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB3 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB4 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU5: CAB5 ffffffffffffffffffffffffffffffffffffffffffffffff
PMU: Error: CA Training Failed.
PMU: ***** Assertion Error - terminating *****
[Result] FAILED