Bit flip errors on 6sl board with LPDDR2 memory

[jayakumar2]

Hi,

I'm using a custom i.mx6sl board. It uses MCIMX6L8DVN10AB with a LPDDR2 (8Gb) 256MBx32 part. I was noticing some sporadic instability during bringup so I ran memtester 100M. The result I got was:

memtester 100M
memtester version 4.3.0 (32-bit)
Copyright (C) 2001-2012 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).

pagesize is 4096
pagesizemask is 0xfffff000
want 100MB (104857600 bytes)
got 100MB (104857600 bytes), trying mlock ...locked.
Loop 1:
Stuck Address : ok
Random Value : ok
Compare XOR : ok
Compare SUB : ok
Compare MUL : ok
Compare DIV : ok
Compare OR : ok
Compare AND : ok
Sequential Increment: ok
Solid Bits : ok
Block Sequential : ok
Checkerboard : ok
Bit Spread : ok
Bit Flip : testing 144FAILURE: 0xffffffff != 0xfffbffff at offset 0x00000780.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x000013c0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0000c440.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0000f180.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00010fc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0001ebc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00020f80.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00022b40.
Walking Ones : ok
Walking Zeroes : ok

Loop 2:
Stuck Address : ok
Random Value : ok
Compare XOR : ok
Compare SUB : ok
Compare MUL : ok
Compare DIV : ok
Compare OR : ok
Compare AND : ok
Sequential Increment: ok
Solid Bits : ok
Block Sequential : ok
Checkerboard : ok
Bit Spread : ok
Bit Flip : testing 144FAILURE: 0xffffffff != 0xfffbffff at offset 0x00005540.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0000dbc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0000dd40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00015f00.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x000212c0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00032700.
Walking Ones : ok
Walking Zeroes : ok

loop 3:

Bit Flip : testing 144FAILURE: 0xffffffff != 0xfffbffff at offset 0x00009f40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0000cd40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00012b40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00015140.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0001d8c0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00031dc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00031e80.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0003ffc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00042c40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0005dec0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00070000.

loop 4:

Bit Flip : testing 144FAILURE: 0xffffffff != 0xfffbffff at offset 0x00009f40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0000cd40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00012b40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00015140.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0001d8c0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00031dc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00031e80.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0003ffc0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00042c40.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0005dec0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00070000.

loop 5:

Bit Flip : testing 144FAILURE: 0xffffffff != 0xfffbffff at offset 0x00018dc0.

loop 6:

Bit Flip : testing 144FAILURE: 0xffffffff != 0xfffbffff at offset 0x000077c0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x000097c0.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x0001df00.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00024180.
FAILURE: 0xffffffff != 0xfffbffff at offset 0x00038d80.

So it looks like I'm getting consistent bit flip errors. I ran DDR calibration (log attached) and switched to the calibrated read/write values.

   Read calibration

   MPRDDLCTL PHY0 (0x021b0848) = 0x3E404646

   Write calibration

   MPWRDLCTL PHY0 (0x021b0850) = 0x3634322A

But rerunning the test with those changes did not cause any change in the stability or result. Rerunning the memtester gave the same bit flip errors.

Does that mean that I have a physical hardware problem? I'm wondering what area of hardware to debug if that's the case.

Thanks in advance for any advice/suggestions.

Original Attachment has been moved to: solace_ddr_calibration_log.txt.zip

[igorpadykov]

Hi Jaya

you are right, this may be due to hardware issues for example with power supplies,

as ddr test does not stress well power subsystem. Please check power supplies

requirements, like ripples provided in i.MX6 System Development User’s Guide

https://www.nxp.com/docs/en/user-guide/IMX6DQ6SDLHDG.pdf

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[jayakumar2]

Hi Igor,

Thanks for your reply. The hardware team suggests that it may still be a software issue and one of the things I'm going to try is to see if the bit flip error occurs at lower DDR clock speed. The other interesting thing is that the exact same bit flip error occurs on multiple boards. They all fail at Bit Flip test 144 with 0xffffffff != 0xfffbffff . I'm wondering if that gives as any additional info on what to debug. It seems to suggest that bit 18 is getting stuck high in the failure sequence.

I'm going to try to work out how to reduce the DDR clock speed for imx6sl to see how that affects the bit flip issue. I'm assuming this is setup in board/freescale/mx6slevk/imximage.cfg . If there's a guide on what registers to watch out for, please let me know.

Thanks!

[larryfeng]

Java, interested to know if you root caused this issue? Thanks.