U-BOOT DDR INITIALIZATION

[yyurtcan]

Hi,

We are using P4080DS demo board.  It has two 2GB Ram. We want to optimize/reduce both bootloader time. For bootloader we use u-boot. While we are working on u-boot, DDR is the one that takes more time.  When we have checked u-boot DDR initialization code timeout is stated 400 milliseconds for each GByte. So, after that point we have done some tests. First of all, we have removed one of DDR3 which is 2GB. We have booted demo board successfully, and check DDR initialization time. It was 200 milliseconds and that elapsed time is for clearing to D_INIT bit. After that, we have attached the removed DDR. We have booted demo board successfully and check DDR initialization time. It was 1000 milliseconds. The 800 milliseconds elapsed time is clearing D_INIT bit for both DDR. And the 200 milliseconds elapsed time is reallocating DDR. After that, we have removed the code that waits for D_INIT bit to be cleared and we have booted demo board successfully when all DDR attached (4GB). We were expecting to pass 200 milliseconds for relocation; however elapsed time was 400 milliseconds. So,

1. 1. What are the reasons for that?
2. 2. What are the reasons that when we update DDR RAM from 2 GByte to 4 GByte it takes much more time (From 200(2GB) + 0(relocation) = 200 millisecond to 400 (2GB ) + 400 (2GB) + 200(relocation) = 1000 millisecond)?
3. 3. How can we reduce DDR initialization time?
4. 4. When we comment out the code that waits D_INIT bir to be cleared, board boots successfully. So, do we really need to wait for D_INIT?

[yyurtcan]

We have discussed about whether we should enable ECC or not. For now, we have decided not to disable ECC due to some reasons this means we cannot run away from that amount of time. There is something that we do not understand when we have only 2GB, it takes 0.223 sn to clear D_INIT bit. However when we have 4 GB, it takes around 0.4318 sn to clear D_INIT bit each of 2GB. We should expect linear time increase however the time is increased squarely. Why is that? 

Best regards.

[Bulat]

You are right, normally you should expect linear time increase... However you have faced with some accidental/unexpected event when the D_INIT proccess takes double time. The problem happens due to DDR controllers interleaving. In such a case CS[0]_BNDS registers of both controllers are set to 4GB size (full memory size). Unfortunately the D_INIT feature is not smart enough, it does not know about second DDR controller and what means interleaving. It writes the entire memory based on CS[0]_BNDS value, i.e. it "thinks" that 4GB is the size of one memory controller (actually still 2GB). So each memory location of each controller is being written two times...

Possible workarounds:

1. To disable DDR controller interleaving.

2. Do not wait for D_INIT as you already tried. In this case both D_INIT procedures (on both controllers) are happening simultaneously, so this takes expected time (about 400ms).

3. To disable DDR controller interleaving and do not wait for D_INIT. This would give further advantage, i.e. will take about 200ms in total.

Regards,

Bulat

[yaseryurtcan]

Hi,

After you have stated that the timing issue we have made maybe be wrong, we have done some tests. Please find the output of each results in the attachment.

1. 1. Test with 2GB  and D_INIT = 1
   1. a. D_INIT_CLEAR = 0.223366 sn
   2. b. Reallocation = 0.266468 sn
2. 2. Test with 4GB  and D_INIT = 1
3. a. D_INIT_CLEAR for one of 2GB = 0.431869 sn
   1. b. D_INIT_CLEAR for one of 2GB = 0.442885 sn
   2. c. Reallocation = 0.265884sn
4. 3. Test with 2GB  and No wait to clear D_INIT bit
   1. a. D_INIT_CLEAR = 0.000748 sn
   2. b. Reallocation = 0.266468 sn
   3. c. But after DDR initcall = 0.199765 sn wait why?
5. 4. Test with 4GB  and No wait to clear D_INIT bit
   1. a. D_INIT_CLEAR for one of 2GB = 0.000759 sn
   2. b. D_INIT_CLEAR for one of 2GB = 0.000737sn
   3. c. But before DDR initcall in tlb_4 = 0.409214 sn wait why?
   4. d. Reallocation = 0.265913 sn
6. 5. Test with 2GB  and D_INIT = 0
   1. a. The system hangs at reallocation.

As you can see from output of each test results, there are some points we did not get. Especially, when we have set D_INIT, the system hangs.  Why is that? And also when we have removed time in order to clear D_INIT bit, the system waits some amount of time. What are the reasons for that?

Best regards.

[Bulat]

As I can see, ECC is enabled in your case. This means that D_INIT must be set (D_INIT=1) to initialize the memory. Otherwise the processor hangs due to multiple ECC errors, this is what you observed.

When D_INIT is enabled, the entire memory is being written after MEM_EN=1. The DDR controller does not run any read/write requests until initialization completes. Or in other words, even if the user does not wait for D_INIT in software, the wait feature woks on HW level, this is what you observed.

Regards,

Bulat

[Bulat]

1,2. I am not sure how exactly you made timing measurements, so accuracy is not so clear.

I would say that relocation time should be absolutely the same in either case, just because the time is defined by the slowest device (i.e. NOR flash) and this device is always the same.

On the "default" P4080DS it takes about 120ms from MEM_EN=1 to D_INIT is cleared for 1GB memory size, so for 2GB it takes about 240ms. As I understand this matches some of your measurements.

3. D_INIT=1 is used to overwrite the whole memory in order to initialize ECC. By default, ECC is disabled on the P4080DS. If it is so in your case, you can simply set D_INIT=0 in the u-boot to exclude ECC initialization. This would significantly reduce initialization time.

4. As I can see in the documentation we never stated that the user needs to wait for D_INIT.

Regards,

Bulat