We are seeing intermittent failures on our pilot run of boards when running the Operational DDR tests in QCVS tool. Our initial run of prototype boards ran through the same suite of tests but did not have this failures. I believe the issue is related to a hardware failure, but I’m having difficulty interpreting the results, and I’m hoping you can shed some light on the output. The tests we are running are as shown below:
The failure can show up in any of the chosen tests but most typically occurs in the DMA Test or the Write-Read-Compare test. The failure that occurs is a failure to initialize, test not run. Here is what the error log is for a failure of the walking zeros test, but the log is basically the same regardless of which test actually failed.
#################### Result for: walking_zeros ###### Run 1 #######################################
Test was not executed!
Err. capture registers:
0xE20, 0x00000000 0xE24, 0x00000000 0xE28, 0x00000000 0xE40, 0x00000080
0xE44, 0x00000000 0xE48, 0x0000001D 0xE4C, 0x00000000 0xE50, 0x00000000
0xE54, 0x00000000 0xE58, 0x00010000
0xF00, 0x00000000 0xF04, 0x00001002 0xF08, 0x0000000B 0xF0C, 0x14000C20
0xF10, 0x00000000 0xF14, 0x00000000 0xF18, 0x00000000 0xF1C, 0x00000000
0xF20, 0x00000000 0xF24, 0x1E002100 0xF28, 0x1B002200 0xF2C, 0x27002D00
0xF30, 0x2A002C00 0xF34, 0x25005000 0xF38, 0x00000000 0xF3C, 0x00000000
0xF40, 0x00000000 0xF44, 0x00000000 0xF48, 0x00000001 0xF4C, 0x94000000
0xF50, 0x0C000F00 0xF54, 0x0B001200 0xF58, 0x16001C00 0xF5C, 0x18001B00
0xF60, 0x14000000 0xF64, 0x00000000 0xF68, 0x00000000 0xF6C, 0x00000000
0xF70, 0x00000000 0xF74, 0x00000000 0xF78, 0x00000000 0xF7C, 0x00000000
0xF80, 0x00000000 0xF84, 0x00000000 0xF88, 0x00000000 0xF8C, 0x00000000
0xF90, 0x00000000 0xF94, 0x00000000 0xF98, 0x00000000 0xF9C, 0x13001300
0xFA0, 0x14001400 0xFA4, 0x12001400 0xFA8, 0x12001400 0xFAC, 0x13000000
0xFB0, 0x00000003 0xFB4, 0x0F0E0E0F 0xFB8, 0x0F0E0E0D 0xFBC, 0x0E0C0C0F
0xFC0, 0x0D0D0D0C 0xFC4, 0x0F0D0E10 0xFC8, 0x0E0E0E0D 0xFCC, 0x0E0D0C0F
0xFD0, 0x0D0D0D0D 0xFD4, 0x0E0C0B0F 0xFD8, 0x0D0C0C0D 0xFDC, 0x0F0E0D10
0xFE0, 0x0E0F0D0E 0xFE4, 0x0E0D0D03 0xFE8, 0x0D0E0E0B 0xFEC, 0x0F0E0D10
0xFF0, 0x0E0E0E0E 0xFF4, 0x0F0D0C0E 0xFF8, 0x0E0E0D0D 0xFFC, 0x00000033
Exception: (<<Error configuring the target! - DDR initialization failed: Invalid value at 0xF04: 0x00001002>>)
Target system was initialized 0 times and it took 0.000000 seconds.
Target system effective test execution took 0.000000 seconds.
Any help deciphering the 0xF04: 0x00001002 output would be appreciated.
Thank you for you assistance with this problem. Adjusting the ODT value seems to have resolved the issue. I believe we have a setting which works for both our prototype and pilot build lots.
I just have one more question regarding the 0xF04 debug register. Where can I find the bit definitions of that register, or can you tell me what the meaning of the bits in the register are? It would be useful to have that information to help debug any future DDR related failures which may occur.
The debug registers are not provided or exposed to customers. The main reason is that it is not really useful information for customer usage.
To debug customer DDR, there are two categories of DDR failures.
1. fail to complete initialization.
2. ECC failures during the operation.
For details on how to tell which categories and how to debug, see the attached FAE training presentation.
Start with a good register settings (you can use QCVS for that).
if we the good setting initialization fails the we have a list of HW issues that customer can examine to see what is causing the failure.
for ECC failures during the operation. use the validation section of QCVS to optimize the setting. if ECC persist with optimized setting then there may be a HW issue (check the AN5097 for the layout guidelines).
We are running more tests, but I believe we have an impedance problem. We need to do some more testing to know for sure if we have a fix or not. I will let you know when we have more information.
Thank you for your prompt response.
I did run through the clock centering, and Read and Write ODT tests. I used the results of the Validation process to create a ".regs" file which is imported when the tool is started, before running the tests.
There is a SPD which was read as part of the project setup. The memory is a Smart Modular SH2047SO410872SC 16GB, DDR4, SO-DIMM.
I set the CodeWarrior connection server for verbose logging as you suggested. The results are in the attached files.
ConnectLog.txt, is the initial connection to the board after turning on verbose logging.
TestLog_DMA_Test_Fail.txt, is the log while the tests were running. In this case the DMA test failed. In an effort to minimize the size of the file I canceled the test after the Write-Read-Compare test ran.
DMA_Test_fail_log.txt, is the log report of the DMA test that failed.
Hello Johh Boggs,
In your error log, DDR_ERR_DETECT(0x00008E40) is read as 0x80, DDR_ERR_DETECT[ACE] is set.
For DDR_ERR_DETECT[ACE] can be set due to the following reasons:
You are using the DDR controller configuration parameters from the SPD, as normal there should be no problem with DDR controller configuration parameters.
I suggest you perform DDR hardware diagnostics.
CodeWarrior provides DDR hardware diagnostics function, please refer to the section "11.3.2 Working with Hardware Diagnostic Action editor" in Freescale\CW_PA_v10.5.1\PA\Help\PDF\Targeting_PA_Processors.pdf.
First please click "C/C++", then create a bareboard CodeWarrior project with "Connect" launch configuration enabled, please modify DDR configuration section in the initialization file in <project_folder>/CFG/T1040RDB_init_core.tcl according to the file ddrCtrl_1.tcl under Generated_Code folder in the QCVS DDRv project. Then connect to the target board to do DDR hardware diagnostics according the user manual.
Hello Johh Boggs,
Have you completed Centering the clock, Read ODT and driver, write ODT and driver test before Operational DDR tests?
Please check whether there is problem with the DDR controller initial configuration parameters. If there is SPD on the target board, please use reading from SPD to create a DDR project. If no, please create a QCVS DDR project with default parameters, then modify Properties panel according to DDR datasheet.
Would you please capture the low level CCS log to me to do more investigation? Please open CodeWarrior Connection Sever console, type "log v", then connect to the target board again, the low level CCS log will be printed in CCS console, would you please capture the CCS log and attach it here?