Custom imx8mm board DDR tool failure with d-cache enabled

[rvanbrenk]

We have created a custom board with a imx8mm processor and a nanya NT6AN128T32AV memory chip of 4Gb.

We started by creating a memory file for the mscale_ddr_too v.3.20 and managed to get a configuration/calibration working, testing the memory is fine as long as we set the  "Disable DDR Memory Cache" option.

When we enable cache we get errors like:

t1: memcpy SSN armv8_x32 test
.Address of test1 failure: 0x000000005000A348
Data was: 0xF7FFFFFF00000200
But pattern was: 0xFFFFFFFF00000200

and 

t1: memcpy SSN armv8_x32 test
.Address of test1 failure: 0x0000000050039F18
Data was: 0xF7FFFFFF00000008
But pattern was: 0xFFFFFFFF00000008
Source is wrong, it is: 0xF7FFFFFF00000008
Address of source failure:0x0000000040039F18

Attached a log of ddr tool.

I do not understand what D-Cache has to do with layout of DDR memory.

[N_Coesel]

Sorry to resurrect this old topic but we still have problems getting the NT6AN128T32AV (512MB LPDDR4) to work reliably with our board. What I'm doing right now is to try to get the memory to run at 1600MHz (which is more than the 1500MHz the IMX8M-mini is specified for but still within specification for the memory) in order to figure out what parameter the memory is sensitive for. I'm getting mixed results though. I'm wondering whether this is a valid approach. My question is: how do I know the memory interface is stable across a batch of boards. Is overclocking the memory interface a good way (if yes, how much?). Or is it better to adjust the ODT / drive strength values? Or should I just trust the memory interface is OK when the stress test runs for 24 hours?

I'm using the DDR stress tool version 3.20 together with RPA v15. It looks like these versions should be OK for the IMX8M-mini?

[N_Coesel]

Meanwhile I have upgraded to RPA22 and using the latest DDR stress test tool. I'm running the memory at 1524MHz during testing to add a little margin for device variations. The iMX8M-mini on our board is the consumer temperature grade. The DDR stress test tool runs OK with the clock frequency set to 1600MHz and the ARM core voltage set to 1V.

When using the memtester tool under Linux I'm getting into a very weird situation.

When locking the maximum frequency of the CPU to 1600MHz and setting the ARM core voltage to 0.90V, I get a memtester failure every 2 loops (so half the test loop fail). Also the board consumes about 3.15W

With the ARM core voltage set to 1.00V, the number of memtester failures drops to less than 10%. BUT the the power consumption also drops to 2.35W.

So when the ARM core voltage is increased, the power consumption lowers by about 0.8W and the memory starts to work much better. How can this happen?

Both tests use the exact same software. Only the kernel DTB file is modified to change the core voltage.

Kernel version: 5.10.72

Other voltages: VDD_SOC = 0.85V, VDD_DRAM = 0.95V, NVCC_DRAM_1V1= 1.1V

Edit:

1) the temperature of the iMX8M-mini is also higher (4 degrees) when the board dissipates more current so the extra power dissipation happens inside the iMX8M-mini as well.

2) When I used the DDR stress test tool, the power consumption is higher with the ARM core voltage set to 1,00V compared to when the ARM core voltage is set to 0,90V.

[N_Coesel]

We finally managed to get the board running stable with the memory at 1500MHz using the latest DDR configuration Excel sheet (in this case version 22) and Linux kernel 5.10.72 . It turns out there are a few problems and pitfalls:

- The DDR stress tool produces a different file (lpdd4_timing.c) for inclusion into uboot depending on whether or not it has run a calibration cycle on the actual hardware. I didn't expect this difference to exist; why isn't the button disabled if the calibration is necessary?

- The busfreq driver messes around with the memory clock causing the instabilities seens under Linux. We have disabled the busfreq driver by changing the source code so it is off by default. It doesn't seem to function well either. Memtest demands a lot from the memory and yet the busfreq driver decides running the memory at a low speed which slows memtest down by about 50%

- We have limited the CPU clock frequency of the commercial version to 1600MHz @ 0.95V. According to the datasheet the CPU in the iMX8M-mini can only run at 1800MHz for 50% of the time?

[rvanbrenk]

We have managed to boot the kernel by only changing the div in u-boot it seems the kernel is not changing the divisor.

[igorpadykov]

Hi

cache is explained in MSCALE_DDR_Tool_User_Guide from link

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX-8M-Family-DDR-Tool-Release/ta-p/110...

One can try different drive strength and ODT settings (search them in RPA file) to see if some combinations allow you to pass

Best regards
igor

[rvanbrenk]

@igorpadykov The drive strength and ODT settings didn't help but lower DDR clocks seem to help so the hardware engineer is reviewing the routing.

In the mean time I want to continue booting the device with a lower DDR speed, I changed the u-boot parameters and memory is trained and initialized fine and reach the U-Boot interactive console.

When booting linux from SD or MMC it loads the DT file and linux but then nothing is happening after the jump to linux. Is Linux kernel re-initializing the DDR memory clock and dividers? And if so can you point me to the correct spot to change it?

[igorpadykov]

>When booting linux from SD or MMC it loads the DT file and linux but then nothing is happening after the >jump to linux. Is Linux kernel re-initializing the DDR memory clock and dividers?

seems linux enables caches so if board did pass ddr test it will also will not boot to linux.

There is no freedom to change ddr clocks, for i.MX8 series ddr can work only at frequencies

defined in RPA tool.

Best regards
igor

[N_Coesel]

Igor,

Unfortunately it seems the board (using an iMX8M-mini SOC) has an issue with running the memory at full speed so I modified the Excel sheet to run the memory at 750MHz (half speed) by setting a different clock divider (just like it is possible with the iMX8MQ devices). I have used the stress testing tool and with this setting and this passes for an overnight test.

So the memory settings seem to be OK. The problem is that the Linux kernel won't boot. The board has 512MB of LPDDR4 memory fitted.

So basically the question is whether the Linux kernel can boot with 512MB of memory or that we need to modify it. Another option is that the kernel does reconfigure the clock dividers for the memory but that seems like an odd thing to do for the Linux kernel.

[igorpadykov]

for 512MB memory size one can try to decrease cma size

https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mm.dtsi?...

also first recommended to try without OPTee, please use sect.5.6.10 OP-TEE enablement

i.MX Yocto Project User’s Guide​

Best regards
igor