UBIFS read only error

sim_guo · ‎09-04-2019

We are testing the iMX6ULL evk. After running for a while, the file system becomes read only, causing all data to be unwritten. This phenomenon has a probability of 1%. Trouble to help analyze the reasons!

Software version: fsl-yocto-L4.1.15_2.0.0-ga

Hardware: i.MX6ULL

Reference design: imx6ulevk (note: use NandFlash (W29N01HVSINA) instead of Emmc)

error log：

ubi0: user volume: 1, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 223/72, WL threshold: 4096, image sequence number: 1809504724
ubi0: available PEBs: 0, total reserved PEBs: 843, PEBs reserved for bad PEB handling: 15
ubi0: background thread "ubi_bgt0d" started, PID 78
dhd_module_init in
snvs_rtc 20cc000.snvs:snvs-rtc-lp: setting system clock to 2019-08-28 06:56:27 UTC (1566975387)
gpio_dvfs: disabling
VSD_3V3: disabling
can-3v3: disabling
ALSA device list:
No soundcards found.
UBIFS (ubi0:0): recovery needed
random: nonblocking pool is initialized
UBIFS (ubi0:0): recovery deferred
UBIFS (ubi0:0): UBIFS: mounted UBI device 0, volume 0, name "rootfs", R/O mode
UBIFS (ubi0:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
UBIFS (ubi0:0): FS size: 103231488 bytes (98 MiB, 813 LEBs), journal size 5206016 bytes (4 MiB, 41 LEBs)
UBIFS (ubi0:0): reserved for root: 4875878 bytes (4761 KiB)
UBIFS (ubi0:0): media format: w4/r0 (latest is w4/r0), UUID C5F22F63-3D0F-4D63-9824-0D27020C3A32, small LPT model
VFS: Mounted root (ubifs filesystem) readonly on device 0:14.
devtmpfs: mounted
Freeing unused kernel memory: 440K (80b1c000 - 80b8a000)
INIT: version 2.88 booting
Starting udev
udevd[107]: starting version 3.1.5
UBIFS (ubi0:0): completing deferred recovery
UBIFS (ubi0:0): background thread "ubifs_bgt0_0" started, PID 127
UBIFS (ubi0:0): deferred recovery completed
bootlogd: cannot allocate pseudo tty: No such file or directory
UBIFS error (ubi0:0 pid 207): ubifs_check_node: bad CRC: calculated 0x61155f3b, read 0xaa0de4ff
UBIFS error (ubi0:0 pid 207): ubifs_check_node: bad node at LEB 774:10936
magic 0x6101831
crc 0xaa0de4ff
node_type 1 (data node)
group_type 0 (no node group)
sqnum 8578819
len 940
key (388974, data, 16)
size 4096
compr_typ 1
data size 892
data:
00000000: 00 4f 65 63 6f 6e 64 73 0d 0a 32 30 31 39 2d 30 38 2d 32 38 20 30 32 3a 32 39 3a 35 37 2e 32 37
00000020: 35 32 38 33 20 5b 49 4e 46 4f 5d 20 5b 69 6f 74 2e 67 61 74 65 77 61 79 2e 77 61 74 63 68 64 6f
00000040: 67 28 70 69 64 3d 31 35 36 39 29 20 28 74 68 72 65 61 64 5f 70 6f 6f 6c 00 00 00 00 00 00 00 00
00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0b 5f 6a 6f 62 5f 6d 67
000000a0: 6d 74 2e 63 3a 34 34 31 29 5d 3a 20 0d 0a 20 20 41 70 70 65 6e 64 77 0e 6a 6f 62 64 0f 0b 70 72
000000c0: 6f 63 65 73 73 20 71 75 65 75 65 2c 68 04 00 04 42 75 73 20 75 6e 69 71 75 65 20 6e 61 6d 65 3a
000000e0: 20 3a 31 2e 33 38 64 03 03 75 75 69 64 3a 20 29 00 00 01 6c 65 64 61 60 03 08 53 65 72 76 69 63
00000100: 65 20 57 4b 4e bc 03 0b 69 6f 74 2e 64 72 69 76 65 72 2e 67 77 5f a4 01 00 02 63 6f 6e 66 69 67
00000120: 5f 75 69 6f 74 5f 6c 69 6e 6b 65 64 67 65 70 07 07 63 6f 75 6e 74 20 64 6f 77 6e ac 07 02 20 31
00000140: 30 20 73 3b 44 06 02 38 31 38 39 39 20 08 49 04 74 91 2f 5f 34 44 06 06 00 00 00 00 00 00 00 00
00000160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0c 06 05 33 37 37 34 35
000001a0: 32 33 32 20 0a 0c 06 02 38 38 37 37 36 20 08 f8 01 20 4c 0c 06 dd 2a 5f bc 4e 78 2b 3b 78 0c 20
000001c0: 02 35 06 35 3c 3c 0c 01 36 38 39 37 20 28 84 12 05 37 31 39 36 32 39 36 30 20 07 2c 06 02 38 2e
000001e0: 33 31 36 65 9a 5b 20 06 80 14 20 36 2e 06 33 35 64 2d 20 15 40 0c 2d 86 12 6c 63 90 9a 2f 50 0c
00000200: 3c 5c 12 60 21 01 38 35 31 37 20 08 50 0a bc 43 36 5c 12 01 34 36 31 33 74 c5 20 0a 1c 06 01 32
00000220: 33 38 37 20 09 f8 01 20 36 1d 06 37 33 18 06 06 6d 61 69 6e 5f 6c 6f 6f 70 74 03 34 38 0c 04 6d
00000240: 70 2e 64 69 6d 75 60 04 30 e4 05 3b 04 0c 04 38 2e 33 32 35 38 38 20 27 34 12 04 31 32 35 37 39
00000260: 34 30 20 08 8c 1e 04 38 2e 33 33 32 38 38 20 60 e6 05 32 32 6c 27 20 14 06 0c 69 6c a4 33 2f e0
00000280: 05 3d 1c 18 6b 1f 33 38 39 20 26 e4 05 06 39 30 34 38 35 34 30 39 36 20 0a cc 0b 02 33 38 39 32
000002a0: 32 20 08 c8 0d 20 36 cc 0b 20 6a 74 24 04 38 2e 33 34 32 35 30 20 09 48 04 3c 14 12 04 38 38 30
000002c0: 37 36 38 38 20 0a 34 18 02 35 31 33 37 30 20 77 2c 0c 01 61 6c 63 73 25 35 2a 5f 68 92 60 66 34
000002e0: 34 12 24 ec 2a 37 5d 0c 38 3a 44 1e 05 38 2e 35 31 35 38 33 34 20 08 58 0a 3c 0c 06 04 32 30 39
00000300: 36 38 30 31 20 0a 54 0c 02 36 35 35 38 35 20 76 74 24 20 04 c8 30 20 02 4c 24 3f 64 12 02 36 35
00000320: 37 30 31 20 27 64 12 04 31 33 32 34 32 37 30 20 08 94 30 04 38 2e 37 30 35 36 34 20 60 dc 36 34
00000340: 94 30 20 4e 84 2a 70 21 01 37 32 37 35 20 26 3c 0c 04 35 34 35 32 32 34 34 20 08 7c 1e 04 39 2e
00000360: 33 32 39 30 36 2a 4c 12 0d 74 2e 67 61 74 65 77 61 79 2e 77 61 74 63 68 64 11 00 00
CPU: 0 PID: 207 Comm: rm Not tainted 4.1.15+g30278ab #1
Hardware name: Freescale i.MX6 Ultralite (Device Tree)
[<80015d78>] (unwind_backtrace) from [<8001271c>] (show_stack+0x10/0x14)
[<8001271c>] (show_stack) from [<80801820>] (dump_stack+0x84/0xc4)
[<80801820>] (dump_stack) from [<8021f3f0>] (ubifs_check_node+0x1b0/0x2e0)
[<8021f3f0>] (ubifs_check_node) from [<802265f8>] (ubifs_scan_a_node+0xc8/0x1e0)
[<802265f8>] (ubifs_scan_a_node) from [<80226abc>] (ubifs_scan+0x58/0x398)
[<80226abc>] (ubifs_scan) from [<8022a970>] (ubifs_garbage_collect_leb+0x70/0x784)
[<8022a970>] (ubifs_garbage_collect_leb) from [<8022b19c>] (ubifs_garbage_collect+0x118/0x3c4)
[<8022b19c>] (ubifs_garbage_collect) from [<80213280>] (make_reservation+0x10c/0x49c)
[<80213280>] (make_reservation) from [<80213968>] (ubifs_jnl_update+0x128/0x5ec)
[<80213968>] (ubifs_jnl_update) from [<80218260>] (ubifs_unlink+0x16c/0x288)
[<80218260>] (ubifs_unlink) from [<800f51dc>] (vfs_unlink+0xb8/0x19c)
[<800f51dc>] (vfs_unlink) from [<800f9960>] (do_unlinkat+0x224/0x244)
[<800f9960>] (do_unlinkat) from [<8000f480>] (ret_fast_syscall+0x0/0x3c)
UBIFS error (ubi0:0 pid 207): ubifs_scan: bad node
UBIFS error (ubi0:0 pid 207): ubifs_scanned_corruption: corruption at LEB 774:10936
UBIFS error (ubi0:0 pid 207): ubifs_scanned_corruption: first 8192 bytes from LEB 774:10936
UBIFS error (ubi0:0 pid 207): ubifs_scan: LEB 774 scanning failed
UBIFS warning (ubi0:0 pid 207): ubifs_ro_mode: switched to read-only mode, error -117
CPU: 0 PID: 207 Comm: rm Not tainted 4.1.15+g30278ab #1
Hardware name: Freescale i.MX6 Ultralite (Device Tree)
[<80015d78>] (unwind_backtrace) from [<8001271c>] (show_stack+0x10/0x14)
[<8001271c>] (show_stack) from [<80801820>] (dump_stack+0x84/0xc4)
[<80801820>] (dump_stack) from [<8022b258>] (ubifs_garbage_collect+0x1d4/0x3c4)
[<8022b258>] (ubifs_garbage_collect) from [<80213280>] (make_reservation+0x10c/0x49c)
[<80213280>] (make_reservation) from [<80213968>] (ubifs_jnl_update+0x128/0x5ec)
[<80213968>] (ubifs_jnl_update) from [<80218260>] (ubifs_unlink+0x16c/0x288)
[<80218260>] (ubifs_unlink) from [<800f51dc>] (vfs_unlink+0xb8/0x19c)
[<800f51dc>] (vfs_unlink) from [<800f9960>] (do_unlinkat+0x224/0x244)
[<800f9960>] (do_unlinkat) from [<8000f480>] (ret_fast_syscall+0x0/0x3c)
UBIFS error (ubi0:0 pid 207): make_reservation: cannot reserve 392 bytes in jhead 1, error -117
rm: can't remove '/var/lib/urandom/random-seed': Structure needs cleaning
urandom start: failed.
INIT: Entering runlevel: 5
Configuring network interfaces... fec 2188000.ethernet eth0: Freescale FEC PHY driver [Micrel KSZ8081 or KSZ8091] (mii_bus:phy_addr=2188000.ethernet:02, irq=-1)
IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
udhcpc (v1.24.1) started
Sending discover...
Sending discover...
Sending discover...

No lease, forking to background
ifconfig: SIOCGIFFLAGS: No such device
[28/Aug/2019:06:56:42 +0000] boa: server version Boa/0.94.13
[28/Aug/2019:06:56:42 +0000] boa: server built Aug 30 2018 at 13:41:51.
[28/Aug/2019:06:56:42 +0000] boa: starting server pid=262, port 80
mkdir: can't create directory '/var/lib/dropbear': Read-only file system
mkdir: can't create directory '/var/lib/dropbear': Read-only file system
cp: can't create '/etc/config/deamon1': File exists
cp: can't create '/zlog.conf': File exists
cp: can't create '/zlog_debug.conf': File exists
cp: can't create '/zlog_error.conf': File exists
Starting syslogd/klogd: cp: can't create '/syslog.conf': File exists
cp: can't create '/syslog.conf-error': File exists

cp: can't create '/syslog.conf-debug': File exists
ntpd: bad address '0.openwrt.pool.ntp.org'
done
mkdir: can't create directory '/db_backup_tar/': File exists
Error: attempt to write a readonly database

Freescale i.MX Release Distro 4.1.15-2.1.0 imx6ulevk /dev/ttymxc0

imx6ulevk login: Error: attempt to write a readonly database
host ver is V3.05.002
init failed
log init failed

igorpadykov · ‎09-05-2019

Hi gxf

this error can be found on many forums (please google it), in particular:

Memory Technology Device (MTD) Subsystem for Linux.

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

sim_guo · ‎09-08-2019

The flash manufacturer thinks the problem is in the master or UBIFS. Is it possible to be a driver problem? Such as the CRC algorithm

igorpadykov · ‎09-08-2019

when ubifs become corrupted and can not recover it becomes read-only.

Suggest to post issue on dedicated forum:

Memory Technology Device (MTD) Subsystem for Linux.

Best regards
igor