iMX6: Destroying SPL on NAND

[kluszon]

I've got a device based on iMX6 with Yocto OS (Dunfell release with kernel: 5.4-2.1 and u-boot: imx_v2018.03_4.14.78). The system normally is starting boot from NAND (SPL, U-Boot, kernel) and the eMMC memory (roofs). The problem I meet is that the device sometimes (totally not regularly) stops working. The touchscreen goes black, diodes on the ethernet port are turned off and debug tty doesn't work. Moreover, first part of mtdblock0 containing SPL is destroyed/changed (example difference_23.10.24.diff, difference_24.10.10.diff) and after the power supply restart, the system can't boot (No correct SPL). It looks like something with main processor.

I don't have any idea what can affect on it. It happens without direct user interaction with the device. I don't see any service on the system side which can modify nand data itself. We checked supply levels, ESD, and magnetic interference on our custom mainboard and we can't reproduce it. Based on the documentation of NAND memory isn't easy to overwrite/erase nand data without an exact command from system side.

Any ideas?

[Zhiming_Liu]

Hi

If re-burn the system, will the same problem occur again? Does your system support ota service? You can try replacing different components to test, such as replacing uboot to exclude the erase command in uboot, or using a pristine busybox filesystem to boot test.

Best Regards
Zhiming

[kluszon]

If re-burn the system, will the same problem occur again?

Unfortunately, yes. It doesn't matter is only a new SPL burn in NAND or the whole system. It happens with a totally random frequency. It happens sometimes in a day another time after a few weeks of working...

Does your system support ota service?

I don't know anything about it. I'm using a custom bash script to update my system.

You can try replacing different components to test, such as replacing uboot to exclude the erase command in uboot, or using a pristine busybox filesystem to boot test.

The problem is that it's random and it's difficult to test any change in the system.

[Zhiming_Liu]

Hi

I think you need to test this in two steps, 1. try using a pristine busybox filesystem to rule out possible footing issues. 2. try upgrading the kernel to rule out possible kernel issues.

Narrow down the problem by testing, and if you can determine that it is caused by a script or service in the filesystem, then you also need to determine step by step which script or service is causing it.

Best Regards
Zhiming