DDRv tool fails due to other DDR hardware or software issues

[Rinke]

Hi,

Im working on the bringup of our own custom board. Its heavily based on the Freeway LS1046A design. The board boots BL2, DDR controller initializes and boots u-boot. All HW is thus working we conclude. Writing to DDR from U-boot works fine. However we would like to tune the ddr settings using the ddrv tool. But it does not want to work. To make sure the settings are all the same i have created a new project and imported the settings from the board. Exporting these settings and putting them into the init file and checking with connection diagnostics results in all checks OK. So DDR settings should be fine right?

When running the ddrv tool it just does not want to work. The first test we want to try and run is the leveling test. See QCVS and ccs log attached. 

What could I be doing wrong that the DDRv tool is not working? I tried to ignore the ACE error to see if I can get past it. But that doesnt solve the issue. Just different output. 

[June_Lu]

Yes, you could read out the uboot values and do the validation, the some bits of wrlv1 wrlvl 2 and 3 would change.

But where the uboot values come from? Many differences from the frwyls1046 values.

Why don't use frwyls1046 value but change the row and col?

What's the DDR data rate?

Thanks

[June_Lu]

From your DDR registers DQ_MAP0, it seems, you have done bit swizzling.

And DDR1_DDR_SDRAM_CFG has obvious error, ECC_EN is not enabled.

So the layout and the DDR part number both different from the Freeway LS1046A design.

Would you provide the info:

- DDR data sheet.

- DDR schematics.

- the snapshot of QCVS "New QorIQ Configuration Guide" -> "DDR configuration" page to check if the DDR SW configurations match the DDR hardware.

Another good reference, after you boot in the uboot, connect the CW read out all related DRAM registers:

(bin) 2 % delete all

(bin) 3 % config cc cwtap:10.193.20.35

(bin) 4 % show cc

0: CodeWarrior TAP (cwtap:10.193.20.35) CC software ver. {0.0}

(bin) 5 % ccs::config_chain {ls1043a dap sap2}

(bin) 6 % ccs::display_mem 32 0x1080100 4 0 4

Read all the related registers,  and compare the registers you configured when do the validation.

[Rinke]

From your DDR registers DQ_MAP0, it seems, you have done bit swizzling.

We dont do bit swizzling. All are connected in order. We have also tried to leave DQ_MAP at default. Does not make a difference. But we tried to set them in order to check if it makes a difference.  I have attached our latest ddr config. This is exported from the board and fully working. Except ddrv then. 

And DDR1_DDR_SDRAM_CFG has obvious error, ECC_EN is not enabled.

When I check ECC_EN is actually enabled in the configuration registers view. 

So the layout and the DDR part number both different from the Freeway LS1046A design.

Yes true in some way its different. But most other parts are the same. Clock design, flash etc. 

Would you provide the info:

- DDR data sheet.

- DDR schematics.

Both see attached

- the snapshot of QCVS "New QorIQ Configuration Guide" -> "DDR configuration" page to check if the DDR SW configurations match the DDR hardware.

Will upload screenshot later

Another good reference, after you boot in the uboot, connect the CW read out all related DRAM registers:

(bin) 2 % delete all

(bin) 3 % config cc cwtap:10.193.20.35

(bin) 4 % show cc

0: CodeWarrior TAP (cwtap:10.193.20.35) CC software ver. {0.0}

(bin) 5 % ccs::config_chain {ls1043a dap sap2}

(bin) 6 % ccs::display_mem 32 0x1080100 4 0 4

Read all the related registers,  and compare the registers you configured when do the validation.

Thanks. I have attached output from ccs ddr registers after uboot booting and after running the wrlvl test. I do see some differences but not sure if any major ones. wrlvl 2 and 3 are set to 0? But this is part of the test I think?

[June_Lu]

Have you followed the QCVS_DDR_User_Guide to configure the DDR parameters.

Have you followed the AN5097, Hardware and Layout Design Considerations for DDR4 SDRAM Memory Interfaces when do the layout?

If your DDR part number and config are all same as Freeway LS1046A design,  you could use the configuration of the Freeway LS1046A first.

Any difference, you should adjust the parameters.

[Rinke]

Hey,

Yes we have followed those guides. Again, the board is able to boot into U-boot with DDR working. So we have a config that should work fine. But using this exact config to run the DDRv tool it does not seem to work. DDR part number is different from Freeway. But again we changed those settings and DDR U-boot is working. So all good. 

Any tips to check why DDRv is not able to be ran?

[June_Lu]

Please confirm your DDR part number and registers you changed per Freeway LS1046A design.

You could share the information here.

 AN5097, Hardware and Layout Design Considerations for DDR4 SDRAM Memory Interfaces should be followed as the layout reference.

[Rinke]

Hi June,

Yes we followed all the guidelines. You can find my exact configuration in the first post. We are using 5x MT40A1G16TB. One for ECC. 8gb of ram in total. Only 1 chip select. 

The configuration we have is working fine. We can boot into u-boot etc. Connection diagnostics works fine.

 

DDR checks pass. Please help me in getting WRLVL tool to work. I dont understand why it hangs on reading register 0x01080114. See first post for full output including ccs logs. 

read_memory(coreh.{serverh=0,cc_index=0,chain_pos=17},
  addr.{addr_hi=0x00000000,addr_lo=0x01080114,size=4,space=4607},count=4,*data)
  data: 00401030

[tzaman]

Juno, have you had the time to review what Rinke sent? In case it helps, this is ddr_init.c we use in ATF:

#include <assert.h>
#include <errno.h>
#include <string.h>

#include <common/debug.h>
#include <ddr.h>
#include <lib/utils.h>

#include <errata.h>
#include <platform_def.h>

const struct ddr_cfg_regs static_1600 = {
	.cs[0].bnds = 0x1FF,
	.cs[0].config = 0x80840512,
	.cs[0].config_2 = 0x00,
	.timing_cfg[0] = 0x80550018,
	.timing_cfg[1] = 0xCDC60F44,
	.timing_cfg[2] = 0x0049111C,
	.timing_cfg[3] = 0x01161000, 
	.timing_cfg[4] = 0x01,
	.timing_cfg[5] = 0x04001000,
	.timing_cfg[7] = 0x23300000,
	.timing_cfg[8] = 0x03336800, 
	.sdram_cfg[0] = 0xE5040008, // 4 stands for 8-beat bursts
	.sdram_cfg[1] = 0x00400000,
	.dq_map[0] = 0x00,
	.dq_map[1] = 0x00,
	.dq_map[2] = 0x00,
	.dq_map[3] = 0x00,
	.sdram_mode[0] = 0x01010210,
	.sdram_mode[8] = 0x0500,
	.sdram_mode[9] = 0x04C80000,
	.md_cntl = 0x00,
	.interval = 0x18600618,
	.data_init = 0xDEADBEEF, 
	.clk_cntl = 0x02800000,
	.init_addr = 0x00, 
	.ddr_sr_cntr = 0x0, 
	.init_ext_addr = 0x00,
	.zq_cntl = 0x8A090705,
	.wrlvl_cntl[0] = 0x86750603,
	.wrlvl_cntl[1] = 0x03080809,
	.wrlvl_cntl[2] = 0x09090908,
	.cdr[0] = 0x80040000,
	.cdr[1] = 0x81,
	.debug[28] = 0x37
};

long long board_static_ddr(struct ddr_info *priv)
{
	memcpy(&priv->ddr_reg, &static_1600, sizeof(static_1600));

	return 0x200000000;
}

long long init_ddr(void)
{
	int spd_addr[] = { NXP_SPD_EEPROM0 };
	struct ddr_info info;
	struct sysinfo sys;
	long long dram_size;

	zeromem(&sys, sizeof(sys));
	get_clocks(&sys);

	debug("platform clock %lu\n", sys.freq_platform);
	debug("DDR PLL1 %lu\n", sys.freq_ddr_pll0);
	debug("DDR PLL2 %lu\n", sys.freq_ddr_pll1);

	zeromem(&info, sizeof(struct ddr_info));
	info.num_ctlrs = 1;
	info.dimm_on_ctlr = 1;
	info.clk = get_ddr_freq(&sys, 0);
	info.spd_addr = spd_addr;
	info.ddr[0] = (void *)NXP_DDR_ADDR;

	dram_size = dram_init(&info);

	if (dram_size < 0)
		ERROR("DDR init failed.\n");

	erratum_a008850_post();

	return dram_size;
}

This code gets us both to u-boot, and further into Linux, which is where stress tests start to fail because obviously, DDR4 is not centred correctly.

[June_Lu]

Please create a new post to address your issue, thanks

[tzaman]

I don't understand. Why do I need to create a new post. Rinke and me are working on the same board, it's the exact same issue, I'm just providing additional information.

[June_Lu]

I am sorry, I find your email address is different from the Rinke's.

And the ddr_init.c file misses the struct dimm_params ddr_raw_timing = {. Which is an important part.

Would you follow my comments 2 weeks before:

You could read out the uboot values and do the validation, the some bits of wrlv1 wrlvl 2 and 3 would change.

But where the uboot values come from? Many differences from the frwyls1046 values.

Why don't use frwyls1046 value but change the row and col?

What's the DDR data rate?

I attached the frwyls1046 CodeWarrior value for you refer.

Please find the LSDK2108 frwyls1046 ddr_init.c  below

https://github.com/nxp-qoriq/atf/blob/LSDK-21.08/plat/nxp/soc-ls1046a/ls1046afrwy/ddr_init.c

It's better to update the register with the mode I send to Rinke.

The CodeWarrior window will also be OK.

Thanks

[tzaman]

Hi June, I think showing you a video will be faster.

As you can see, I can successfully boot into Linux on the board we're bringing up. Reset works properly as well.

Then I load the DDR register values from this very same running board and try validation, which fails.

I don't understand we need the other part (dimm_params ddr_raw_timing), we're using the static configuration so this part doesn't get compiled in.

Interestingly enough, if I take the generated code that CodeWarrior gives me based on this configutation, and in fact use what you're suggesting (set CONFIG_STATIC_DDR to 0), the board doesn't boot. The generated code is quite confusing, and likely wrong.

So the question remains:

Why can we boot into Linux on this board (over and over again, it's not a one-off thing), but when trying to validate DDR, not a single value gets successfully written to DDR?

Thank you.

[tzaman]

One further observation. I looked at the process that validation tool uses to do the write leveling tests (and all others I assume). 

Once it resets the controller, it checks register 0x01080F04 and in the case of RDB (which we have), this register returns 0x00000002 and the validator then proceeds to test the memory.

In our case, on the other hand, the register 0x01080F04 returns 0x00003002 which I believe makes the tool not to proceed with actual tests. Is there a way for us to learn what this register means?

Here's the debug register values that I get when I read them in u-boot, on the actual board that DDR centering tests are failing:

=> md 0x01080F00
01080f00: 00000000 02300000 0d000000 200c0014  ......0........
01080f10: 00000000 00000000 00000000 00000000  ................
01080f20: 00000000 10101010 10101010 10101010  ................
01080f30: 10101010 00301010 00000000 00000000  ......0.........
01080f40: 00000000 00000000 01000000 000000d4  ................
01080f50: 000b000c 000f000e 00120012 00140012  ................
01080f60: 00000010 00900000 20000000 00000000  ........... ....
01080f70: 37007000 00000000 00000000 00000000  .p.7............
01080f80: 00000000 00000000 00000000 00000000  ................
01080f90: 00000000 00000080 00000000 0030002f  ............/.0.
01080fa0: 002f0030 0030002f 0030002e 0000002f  0././.0...0./...
01080fb0: 03000000 1d1c1c1f 0e1e1d0e 0f0f1d1f  ................
01080fc0: 1d1e1e1e 1e1d1e1f 1d1e1d1d 1f1e1f1f  ................
01080fd0: 1d1f1f1e 1d1a1d1f 1b1c1e1c 1e1b1e1f  ................
01080fe0: 1d1e1f1e 0d1c0d1f 1b1c1e1d 1f1c1f1f  ................
01080ff0: 1d1f201e 1c1a1e1f 1b1c1e1c 0033001f  . ............3.

You can see F04 being set to 02300000, which is the same value, written in little endian. 

Is there a way we could investigate this register, or have an idea why it is set to this particular value?

And please don't respond with "F04 is not a public register" because that gives me nothing to investigate further.

Thanks!

[tzaman]

I've gotten a bit further in my understanding.

If in u-boot, I check 0x01080E40, I can see that ACE bit is set (which is why I believe F04 is set the way it is, but I might be wrong):

=> md 0x01080E40
01080e40: 80000000 00000000 1d000000 00000000  ................

I was under the impression that ACE bit being set would prevent u-boot (let alone Linux kernel) to successfully boot.

This is even more confusing because we can successfully write and read to any offset on the memory, in u-boot, from 0-8GB and get a successful match.

Please advise.

[gkrishna]

Hi Rinke,

I am working in the similiar design but mine is not getting into the U-boot and it stalls when starting the u-boot in DDR. STill I am missing something the DDR parameters. Could you please tel me what and all tha params you modified/considered for DDR init in Bl2 Code? I know this might be wrong thread to ask this but please do help.