DDR initialization failure on custom LS1046A-based board

DN31415 — Wed, 28 Jul 2021 07:31:28 GMT

Summary

We're attempting to bring up a custom board based on the NXP LS1046A SoC. We're currently struggling with DDR initialization errors in BL2 (ATF). After experimenting with the DDR controller settings, we see one of the following three outcomes:

DDR initialization and training fails with error code (debug[1] value of) 0x3002. We are unable to find any documentation on the meaning of the error bits.
DDR initialization and training times out (D_INIT is not released).
DDR initialization and training appears to succeed, but BL3 fails to load, with error messages indicative of memory malfunction:
- ERROR: mmap_add_region_check() failed. error -22
- ERROR: SD read error - DMA error = 10000000
- ERROR: Read error = fffffffb
- WARNING: Failed to obtain reference to image id=3 (-2)
- ERROR: BL2: Failed to load image (-2)
- Authentication failure

Hardware Overview

SoC part: LS1046AXE8P1A.
Discrete SDRAM part: 2x MT40A1G16RC-062E IT:B, single chip select, 32-bit data width (16 bits per component).
Boot source: micro-SD card.
Clock input: DIFF_SYSCLK is a 100 MHz AC-LVDS signal from an LMK05318B PLL. It conforms to the requirements in section 3.7.6 of the LS1046A Datasheet. SYSCLK and DDRCLK are not used and are tied to GND.
SoC strapping:
- CFG_DRAM = 0 = DDR4
- CFG_ENG_USE0 = 0 = DIFF_SYSCLK
- CFG_ENG_USE1 = 1 = Use internal LVDS termination
- CFG_RCW_SRC[8..0] = 0_0100_0000 = Boot from SD/MMC
RCW settings:
- 400 MHz platform clock (SYS_PLL_RAT=4)
- 1600 MHz DDR clock (MEM_PLL_RAT=16)
- 1200 MHz PLL1 (CPU) (CGA_PLL1_RAT=12)
- 1200 MHz CPU core clock from PLL1 /1 (C1_PLL_SEL=0)
- DDRCLK not used (DDR_REFCLK_SEL=1)
- DDR_FDBK_MULT=2
- DRAM_LAT=1
- PBI_src=6
- SB_EN=0

What We've Checked / Tried

Hardware

All SoC input voltages, clocks and resets have been checked against the LS1046A datasheet specifications for levels and timing.
The 800 MHz DDR bus clock (generated by the SoC) is present and there is activity on the DDR control signals.

Firmware

The QCVS DDR configuration settings have been checked against the MT40A1G16RC-062E IT:B datasheet.
All relevant CCSR registers have been checked in ATF to ensure correct strapping and RCW values.
Several recent versions of the ATF firmware from the qoriq-components Git repository have been tested.
The DDR configuration register values printed on the console have been checked against the intended settings from QCVS's generated code, and all match.
We confirmed that the workaround code for errata A008511, A009803, A009942 and A010165 have been applied.
We tried a few variations of platform clock frequency and DDR controller frequency.
We tried a few variations of the CLK-to-DQS skew values. (The values shown in the QCVS screenshot below are the real board values from Altium.)

Reference Data

RCW Source File

#include <ls1046a.rcwi>

/* clocking */

SYSCLK_FREQ=600 /* SYSCLK input = 600 * 166.667 kHz = 100 MHz */

SYS_PLL_RAT=4 /* platform clock = 4 * SYSCLK = 400 MHz */

MEM_PLL_RAT=16 /* DDR clock = 16 * SYSCLK = 1600 MHz */

CGA_PLL1_RAT=12 /* PLL1 (CPU) = 12 * SYSCLK = 1200 MHz */

CGA_PLL2_RAT=12 /* PLL2 (FMan, eSDHC, QSPI) = 12 * SYSCLK = 1200 MHz */

C1_PLL_SEL=0 /* CPU core clock = PLL1 /1 = 1200 MHz */

HWA_CGA_M1_CLK_SEL=6 /* FMan clock = PLL2 /2 = 600 MHz */

HWA_CGA_M2_CLK_SEL=1 /* eSDHC*6, QSPI*64 clocks = PLL2 /1 = 1200 MHz */

DDR_REFCLK_SEL=1 /* use differential SYSCLK, not DDRCLK, for DDR PLL */

/* I/O bank voltages */

TVDD_VSEL=1 /* TVDD I/O domain: 2.5 V */

DVDD_VSEL=2 /* DVDD I/O domain: 3.3 V */

EVDD_VSEL=2 /* EVDD I/O domain: 3.3 V */

/* SerDes 1 settings */

SRDS_PRTCL_S1=4403 /* 0x1133: 0=XFI.9, 1=XFI.10, 2=SGMII.5, 3=SGMII.6 */

SRDS_REFCLK_SEL_S1=0 /* use separate reference clocks for PLL1 and PLL2 */

SRDS_PLL_PD_S1=2 /* power down PLL1 */

SRDS_PLL_REF_CLK_SEL_S1=3 /* PLL1 @ N/A, PLL2 @ 156.25 MHz */

/* SerDes 2 settings */

SRDS_PRTCL_S2=21879 /* 0x5577: 0=PCIe.1 x1, 1=PCIe.2 x1, 2/3=PCIe.3 x2 */

SRDS_REFCLK_SEL_S2=0 /* use separate reference clocks for PLL1 and PLL2 */

SRDS_PLL_REF_CLK_SEL_S2=3 /* PLL1 @ 100 MHz, PLL2 @ 100 MHz */

SRDS_DIV_PEX_S2=0 /* PCIe can train up to 8 Gb/s */

/* memory settings */

DDR_FDBK_MULT=2 /* (reserved) DDR PLL feedback path selection and multiplication enabler */

DRAM_LAT=1 /* 8-8-8 or higher latency DRAMs */

/* boot settings */

PBI_src=6 /* fetch pre-boot initialization data from SD/MMC */

SB_EN=0 /* secure boot is not enabled */

/* pin multiplexing */

IFC_GRP_A_EXT=1 /* use QSPI A signals, not FTM5, GPIO or IFC */

IFC_GRP_D_EXT=1 /* use QSPI B signals, not FTM6, GPIO or IFC */

IFC_GRP_E1_BASE=1 /* use GPIO2 signals, not FTM7, IFC or QSPI B DATA3 */

IFC_GRP_F_EXT=1 /* use QSPI A signals, not IFC */

IIC2_EXT=1 /* use SDHC_CD_B and SDHC_WP, not IIC2/GPIO/FTM */

IRQ_OUT=1 /* (reserved) */

IRQ_BASE=1 /* use GPIO, not IRQ */

UART_BASE=4 /* use UART1 SOUT/SIN/RTS/CTS and GPIO1_[16:22]*/

USB_DRVVBUS=1 /* use GPIO4[29], not USB_DRVVBUS */

USB_PWRFAULT=1 /* use GPIO4[30], not USB_PWRFAULT */

EC1=5 /* use FTM1 */

EC2=5 /* use FTM1 */

EM1=1 /* use GPIO_3 */

EM2=1 /* use GPIO_4 */

/* Pre-Boot Initialization (PBI). This PBI code mostly derives

* from the LS1046A RDB sample (rcw_1800_emmcboot.rcw). */

.pbi

// SCFG_SCRATCHRW[1:2]: 64-bit boot pointer

// 0x10000000 is 64 KiB on-chip RAM (OCRAM1); PBL loads BL2 here

write 0x570600, 0x00000000

write 0x570604, 0x10000000

.end

#include <cci_barrier_disable.rcw> // CCI: all master interfaces terminate barriers (why?)

#include <usb_phy_freq.rcw> // take USB1, USB2 and USB3 PHYs out of reset

#include <pex_gen3_link.rcw> // PCIe errata A-010477, A-008851 (re: Gen 3)

#include <a009531.rcw> // PCIe erratum A-009531 (re: ID0 bit)

DDR Initialization Code in ATF

DDR initialization code was generated by the latest version of QCVS running in CodeWarrior 11.5.0 on a Windows PC.

const struct ddr_cfg_regs static_1600 = {
    .cs[0].bnds = 0xFF,
    .cs[1].bnds = 0x0100013F,
    .cs[0].config = 0x80010512,
    .cs[1].config = 0x0202,
    .cs[2].bnds = 0x0140017F,
    .cs[3].bnds = 0x018001BF,
    .cs[2].config = 0x0202,
    .cs[3].config = 0x0202,
    .cs[0].config_2 = 0x00,
    .cs[1].config_2 = 0x00,
    .cs[2].config_2 =  0x00,
    .cs[3].config_2 =  0x00,
    .sdram_cfg[0] = 0xC52C0000,
    .sdram_cfg[1] = 0x00401030,
    .timing_cfg[0] = 0xFF550018,
    .timing_cfg[1] = 0xBAB40C52,
    .timing_cfg[2] = 0x0048D11C,
    .timing_cfg[3] = 0x01161000,
    .timing_cfg[4] = 0xD501,
    .timing_cfg[5] = 0x03401400,
    .timing_cfg[7] = 0x23340000,
    .timing_cfg[8] = 0x02116600,
    .dq_map[0] = 0x32C57554,
    .dq_map[1] = 0xD4BB0BD4,
    .dq_map[2] = 0x2EC2F554,
    .dq_map[3] = 0xD8000000,
    .sdram_mode[0] = 0x01010210,
    .sdram_mode[1] = 0x00,
    .sdram_mode[2] = 0x00,
    .sdram_mode[3] = 0x00,
    .sdram_mode[4] = 0x00,
    .sdram_mode[5] = 0x00,
    .sdram_mode[6] = 0x00,
    .sdram_mode[7] = 0x00,
    .sdram_mode[8] = 0x0701,
    .sdram_mode[9] = 0x04800000,
    .sdram_mode[10] = 0x00,
    .sdram_mode[11] = 0x00,
    .sdram_mode[12] = 0x00,
    .sdram_mode[13] = 0x00,
    .sdram_mode[14] = 0x00,
    .sdram_mode[15] = 0x00,
    .md_cntl = 0x00,
    .interval = 0x18600618,
    .data_init = 0xDEADBEEF,
    .clk_cntl = 0x02800000,
    .init_addr = 0x00,
    .ddr_sr_cntr = 0x0,
    .init_ext_addr = 0x00,
    .zq_cntl = 0x8A090705,
    .wrlvl_cntl[0] = 0x86750606,
    .wrlvl_cntl[1] = 0x0606060B,
    .wrlvl_cntl[2] = 0x0B0C0C00,
    .cdr[0] = 0x80040000,
    .cdr[1] = 0x81,
};

Console Output

INFO:    RCW BOOT SRC is SD/EMMC
INFO:    RCW BOOT SRC is SD/EMMC
INFO:    esdhc_emmc_init
INFO:    Card detected successfully
INFO:    init done:
INFO:    platform clock 400000000
INFO:    DDR PLL1 1600000000
INFO:    DDR PLL2 0
NOTICE:  Using static DDR settings
NOTICE:  PORSR1: 207f7fff
NOTICE:  PORSR2: cf000000
NOTICE:  CLKCCSR: 0
NOTICE:  CL1KCGHWACSR: 30000000
NOTICE:  CL2KCGHWACSR: 8000000
NOTICE:  PLLC1GSR: 18
NOTICE:  PLLC2GSR: 18
NOTICE:  CLKPCSR: 0
NOTICE:  PLLPGSR: 8
NOTICE:  PLLDGSR: 80020
INFO:    Time before programming controller 30 ms
INFO:    Program controller registers
WARNING: Warning: Optimal CPO value not set.
INFO:    total size 4 GB
INFO:    Need to wait up to 2680 ms
INFO:    Reading debug[9] as 0x10101010
INFO:    Reading debug[10] as 0x10101010
INFO:    cpo_min 0x10
INFO:    cpo_max 0x10
INFO:    debug[28] 0x70006f
WARNING: Warning: A009942 requires setting cpo_sample to 0x37
ERROR:   Found training error(s): 0x3002
INFO:    *0x1080000 = 0xff
INFO:    *0x1080008 = 0x100013f
INFO:    *0x1080010 = 0x140017f
INFO:    *0x1080018 = 0x18001bf
INFO:    *0x1080080 = 0x80010512
INFO:    *0x1080084 = 0x202
INFO:    *0x1080088 = 0x202
INFO:    *0x108008c = 0x202
INFO:    *0x1080100 = 0x1161000
INFO:    *0x1080104 = 0xff550018
INFO:    *0x1080108 = 0xbab40c52
INFO:    *0x108010c = 0x48d11c
INFO:    *0x1080110 = 0xc52c0000
INFO:    *0x1080114 = 0x401020
INFO:    *0x1080118 = 0x1010210
INFO:    *0x1080120 = 0x600041f
INFO:    *0x1080124 = 0x18600618
INFO:    *0x1080128 = 0xdeadbeef
INFO:    *0x1080130 = 0x2800000
INFO:    *0x1080160 = 0xd501
INFO:    *0x1080164 = 0x3401400
INFO:    *0x108016c = 0x23340000
INFO:    *0x1080170 = 0x8a090705
INFO:    *0x1080174 = 0xc6750606
INFO:    *0x1080190 = 0x606060b
INFO:    *0x1080194 = 0xb0c0c00
INFO:    *0x1080220 = 0x701
INFO:    *0x1080224 = 0x4800000
INFO:    *0x1080250 = 0x2116600
INFO:    *0x1080270 = 0xffff
INFO:    *0x1080280 = 0xffffff77
INFO:    *0x1080284 = 0xffffff77
INFO:    *0x1080288 = 0x88888800
INFO:    *0x108028c = 0xffffffff
INFO:    *0x1080290 = 0x1
INFO:    *0x10802a0 = 0x1
INFO:    *0x1080400 = 0x32c57554
INFO:    *0x1080404 = 0xd4bb0bd4
INFO:    *0x1080408 = 0x2ec2f554
INFO:    *0x108040c = 0xd8000000
INFO:    *0x1080b20 = 0x8080
INFO:    *0x1080b24 = 0x80000000
INFO:    *0x1080b28 = 0x80040000
INFO:    *0x1080b2c = 0x81
INFO:    *0x1080bf8 = 0x20502
INFO:    *0x1080bfc = 0x100
INFO:    *0x1080e40 = 0x80
INFO:    *0x1080f04 = 0x3002
INFO:    *0x1080f08 = 0xe
INFO:    *0x1080f0c = 0x14000c20
INFO:    *0x1080f24 = 0x10101010
INFO:    *0x1080f28 = 0x10101010
INFO:    *0x1080f2c = 0x10101010
INFO:    *0x1080f30 = 0x10101010
INFO:    *0x1080f34 = 0x10103000
INFO:    *0x1080f48 = 0x1
INFO:    *0x1080f4c = 0x94000000
INFO:    *0x1080f50 = 0xd000c00
INFO:    *0x1080f54 = 0xb000b00
INFO:    *0x1080f58 = 0x16001600
INFO:    *0x1080f5c = 0x18001800
INFO:    *0x1080f60 = 0xc000000
INFO:    *0x1080f64 = 0x9000
INFO:    *0x1080f68 = 0x20
INFO:    *0x1080f70 = 0x70006f
INFO:    *0x1080f94 = 0x80000000
INFO:    *0x1080fb0 = 0x3
INFO:    *0x1080fb4 = 0x1f1f1f1f
INFO:    *0x1080fb8 = 0x1f1f1f1f
INFO:    *0x1080fbc = 0x1f1f1f1f
INFO:    *0x1080fc0 = 0x1f1f1f1f
INFO:    *0x1080fc4 = 0x1f1f1f1f
INFO:    *0x1080fc8 = 0x1f1f1f1f
INFO:    *0x1080fcc = 0x1f1f1f1f
INFO:    *0x1080fd0 = 0x1f1f1f1f
INFO:    *0x1080fd4 = 0x1f1f1f1f
INFO:    *0x1080fd8 = 0x1f1f1f1f
INFO:    *0x1080fdc = 0x1f1f1f1f
INFO:    *0x1080fe0 = 0x1f1f1f1f
INFO:    *0x1080fe4 = 0x1f1f1f1f
INFO:    *0x1080fe8 = 0x1f1f1f1f
INFO:    *0x1080fec = 0x1f1f1f1f
INFO:    *0x1080ff0 = 0x1f1f1f1f
INFO:    *0x1080ff4 = 0x1f1f1f1f
INFO:    *0x1080ff8 = 0x1f1f1f1f
INFO:    *0x1080ffc = 0x1f001f18
ERROR:   Writing DDR register(s) failed
ERROR:   Programing DDRC error
ERROR:   DDR init failed.
NOTICE:  Incorrect DRAM0 size is defined in platfor_def.h
ERROR:   mmap_add_region_check() failed. error -22
ERROR:   mmap_add_region_check() failed. error -22
NOTICE:  BL2: v1.5(debug):LSDK-19.09-update-311219-dirty
NOTICE:  BL2: Built : 13:08:45, Jul 27 2021
INFO:    Configuring TrustZone Controller
INFO:    Value of region base = 7fdffffb
INFO:    Value of region base = 7ffffffb
INFO:    Value of region base = fbdffffb
INFO:    BL2: Doing platform setup
INFO:    BL2: Loading image id 3
ERROR:   SD read error - DMA error = 10000000
ERROR:   Read error = fffffffb
WARNING: Failed to obtain reference to image id=3 (-2)
ERROR:   BL2: Failed to load image (-2)
Authentication failure

Re: DDR initialization failure on custom LS1046A-based board

DN31415 — Thu, 29 Jul 2021 07:01:26 GMT

We have solved this issue. The cause was a combination of two factors:

1. QCVS by default generates a DQ swizzling/remapping pattern in the dq_map registers. We don't have any swizzling on the board, so we had to set dq_map[0..3] to 0 to set a 1:1 bit mapping. There is another thread about this here: https://community.nxp.com/t5/CodeWarrior-for-QorIQ/DDR4-Memory-Validation-Error-Message/m-p/606956

2. According to erratum A007864 there are problems running 16-bit data widths above 1333 MT/s. We have a 32-bit data width (2x ICs with 16 bits per IC) and it seems this also meets the criteria of A007864, as DDRv failed until we reduced the bus speed to 1300 MT/s.

I am curious why QCVS by default generates such strange DQ remapping values.

Re: DDR initialization failure on custom LS1046A-based board

zhangthink — Thu, 09 Feb 2023 01:25:31 GMT

@DN31415

We meet the same problem as we use MT40A1G8SA:062E，but I can't find the erratum A007864 as you refered in before replies, may you provide the erratum A007864 to me, Thanks!

Re: DDR initialization failure on custom LS1046A-based board

DN31415 — Thu, 09 Feb 2023 08:36:04 GMT

Hi @zhangthink,

The errata can be found in the Documentation section for your SoC. I can't provide the document because NXP now requires an NDA and the document requires a Sharepoint invitation. I'm not positive but I believe this is a new policy and the document used to be publicly available...

If you need an NDA / invitation you will need to contact NXP, either here or via your silicon supplier.

Regards

Re: DDR initialization failure on custom LS1046A-based board

tzaman — Tue, 15 Apr 2025 15:47:08 GMT

Hey DN31415 - we are using the exact same part and see exact same issues. However, if in DDR configuration tool, I try to go below 1600MT/s, I get an error. If possible, would you be able to send me your working DDR4 registers so I can compare what I'm doing wrong?

Thank you!

Re: DDR initialization failure on custom LS1046A-based board

DN31415 — Wed, 16 Apr 2025 14:34:39 GMT

Hello,

My original NXP account seems to have been deleted, so I am replying under a newly-created account with a different username.

I don't have the CodeWarrior project for this board anymore, but I can provide you with the generated ddr_init.c file which is compiled into ATF. I have attached the file to this post. The file contains the DDR controller registers in the ddr_cfg_regs structure.

In addition, we changed the MEM_PLL_RAT field (DDR clock) in the .rcw file to 13 (meaning 13 * SYSCLK, yielding 1300 MHz assuming SYSCLK is 100 MHz).

Please double-check the other DDR controller settings in this file for compatibility with your board. I hope this helps!

Best regards,

David Norris

topic Re: DDR initialization failure on custom LS1046A-based board in Layerscape

DDR initialization failure on custom LS1046A-based board

Summary

Hardware Overview

What We've Checked / Tried

Hardware

Firmware

Reference Data

RCW Source File

DDR Initialization Code in ATF

Console Output

Re: DDR initialization failure on custom LS1046A-based board

Re: DDR initialization failure on custom LS1046A-based board

Re: DDR initialization failure on custom LS1046A-based board

Re: DDR initialization failure on custom LS1046A-based board

Re: DDR initialization failure on custom LS1046A-based board