i.MX8QM DDR suspend/resume issue

[franz86]

I would like to report an issue we have with a custom designed i.MX8QM board with the following specs:

• i.MX 8QuadMax (IMX8QMRM) SoC
• DDR4 RAM: 4GB, 1.6GHz (Samsung K4F6E3S4HM-GHCL)
• eMMC Flash: 16GB
• PMICs 2x PF8200

Relevant software components:

• SCFW 6638c032 (based on 1.15.0, imx-sc-firmware_1.15.0.bb, meta-freescale, kirkstone branch)
• SECO-FW c9de51c0
• IMX-MKIMAGE 3bfcfccb
• ATF 3c1583b
• U-Boot 2022.04-ttc
• Linux Kernel 5.15.87-5.15.87-2.2.0+gc9c3669fe875 (based on meta-freescale kirkstone branch)

We encountered a problem related to the DDR when waking up from suspend mode:

During the testing of the deep sleep mode (Suspend2RAM) we found out, that when we're trying to resume the board from suspend with a previous configured wake-up source e.g. UART wakeup, the board reboots.

Steps to reproduce:

1. echo enabled > /sys/class/tty/ttyLP0/power/wakeup
2. systemctl suspend
3. # wait for board to suspend
4. # type anything into the serial console
5. # board reboots after ~2 seconds

While debugging, we found out that in the phase of resuming, the System Control Unit (SCU) is stuck for about two seconds and afterwards the SCU_WDOG_OUT signal goes to HIGH and causes a reboot.

When the board goes to suspend mode, we saw with the debugger that the SCU reads PHY register values and stores them into the RAM. When resuming from suspend those PHY register values are read back from the RAM and will be written to the PHY registers (These values are actually values that are taken from the boot phase, from the RAM configuration). During this phase it happens that when a value is written to the PIR (DDR_PHY_PIR_0) register the status register signals an error.

The error that is set is the "Write Leveling Adjustment Error". When this bit is set the SCU seems to jump to a while(true) loop and waits until the watchdog triggers.

During weeks of debugging we did HW simulations, we ran the SCU stress tester, we also checked with the i.MX8 vTSA tool and generated DDR eye-diagrams, but none of those revealed any issues on our board layout. All results showed that we are within the margins. So at this point we can say with a high probability that there is no issue in our board design/layout.

Can anyone from NXP or even better: someone from the SCFW development team please explain what causes could lead to such a Write Level Adjustment Error? What measures can be taken to avoid this? Could it be possible that this is an issue in the SCFW firmware version that we are using?

One additional note: We saw, that the issue only occurs when the DDR frequency is clocked at 1600MHz. When using 1200MHz the issue is gone.

Any helpful information would be highly appreciated!

[sdarley]

For support team:  I have created a duplicate ticket in a different forum (with the same problem title) and attached some additional information that include detailed screenshots captured when debugging and a workaround.  Please contact me if you have trouble finding it.