Kernel panic when login on iMX6 (Micron Nand + ubifs)

[abrahim]

Hello,

I'm building a buildroot BSP for a iMX6 solo core :

• U-Boot 2020.04
• Linux version 5.10.9
• Buildroot 2022.11-1380-g1613273542

When I boot and login, I have a long kernel panic message. It seems to be due to the filesystem :

Buildroot login: root
[ 16.615654] 8<--- cut here ---
[ 16.618754] Unable to handle kernel paging request at virtual address 0124d494
[ 16.626708] 8<--- cut here ---
[ 16.629793] Unable to handle kernel NULL pointer dereference at virtual address 00000003
[ 16.637996] pgd = 302756ad
[ 16.640719] [00000003] *pgd=00000000
[ 16.644353] Internal error: Oops: 805 [#1] PREEMPT SMP ARM
[ 16.649856] Modules linked in: imx_vdoa
[ 16.653720] CPU: 0 PID: 257 Comm: systemd Not tainted 5.10.9 #39
[ 16.659744] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 16.666300] PC is at __insert_inode_hash+0x7c/0x94
[ 16.671099] LR is at __insert_inode_hash+0x5c/0x94
[ 16.675903] pc : [<80272dc8>] lr : [<80272da8>] psr: a00b0013
[ 16.682185] sp : 8296bc78 ip : 0003df3f fp : 8429e198
[ 16.687421] r10: 8444a678 r9 : 00000048 r8 : dabf7878
[ 16.692650] r7 : 844ba068 r6 : dab99000 r5 : 844ba0a0 r4 : 00017a1e
[ 16.699184] r3 : ffffffff r2 : 00000001 r1 : 00000000 r0 : 844ba068
[ 16.705727] Flags: NzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 16.712879] Control: 10c5387d Table: 1288c059 DAC: 00000051
[ 16.718638] Process systemd (pid: 257, stack limit = 0xa7b5c38d)
[ 16.724660] Stack: (0x8296bc78 to 0x8296c000)
[ 16.729030] bc60: 00000000 8429e000
[ 16.737233] bc80: 8444a660 82671000 844ba000 803efcf8 00000000 00000000 827a70c0 00000000
[ 16.745432] bca0: 00100050 00000000 00000180 000001d8 000000a0 8444a678 8444a68c 00000008
[ 16.753630] bcc0: 00000000 00000000 00000000 00000000 00000000 ba1e2c76 8296bd70 803efb34
[ 16.761829] bce0: 8444a660 841d26e8 00000000 8296bd70 8296be18 827a70c0 8120fec4 80266750
[ 16.770027] bd00: 00000001 00000005 82772000 80249ecc 00000040 000280c2 00000001 00000000
[ 16.778217] bd20: 8429e000 00008180 00000040 8296a000 00000001 ffffffec 00000000 8296bd3c
[ 16.786408] bd40: 8296bd3c ba1e2c76 00000000 000280c2 8296be18 00000000 8139ddb8 00000000
[ 16.794608] bd60: 829b9200 00000000 8296a000 8026866c 826943d0 841d26e8 61f080fd 00000008
[ 16.802800] bd80: 82a00010 82713d04 00000000 8024bac0 8429e000 00000700 00000002 000008de
[ 16.810999] bda0: 000004ac 00000000 00000000 00000000 8296bdb4 8296a000 8296bdc8 0008b420
[ 16.819197] bdc0: 82a00000 80242130 8296bdd0 00000000 82a00000 ba1e2c76 ffffffec 000280c2
[ 16.827396] bde0: 00000008 82a00000 00000000 829b9200 ffffff9c 00000000 82a041ed 80267034
[ 16.835587] be00: 000280c2 ba1e2c76 000280c2 82a00000 00000180 80252920 000280c2 828c8180
[ 16.843777] be20: 00000006 00000700 00000000 ba1e2c76 828c4300 82938a00 812df550 802c7a20
[ 16.851968] be40: 8c01818f ffefe0c8 8c01818f 801164c4 00000000 00000000 00000000 00000000
[ 16.860159] be60: 828c4f00 828c4300 00000000 829b9200 00000008 00000080 826dfd20 80242130
[ 16.868356] be80: 8296be88 8296a000 826dfd20 00000000 826ff6c0 00000000 00000000 00000000
[ 16.876555] bea0: 8296bea0 8296bea0 8296bf60 8296bfb0 00000000 ffffffff 000000cd 00000000
[ 16.884752] bec0: 00000000 00000000 00000000 00000000 7ed860ec ba1e2c76 00000001 0000000a
[ 16.892944] bee0: 8296a000 400004d8 8294ca80 8296bf60 826c9180 8296a000 0000000a 801401a4
[ 16.901135] bf00: 00000000 8296bf4c 826943c0 81395028 8294ca80 8296a000 8296bfb0 76a64d28
[ 16.909326] bf20: fffffe30 8296bf4c 5ac3c35a 81204f8c 00000000 8010984c 0000000b 76a64d2c
[ 16.917516] bf40: 8296bf64 8013f904 00000000 00000101 0000000b 8296a000 ffffe000 80140e7c
[ 16.925713] bf60: 0000000b 00000000 00000000 00000101 00000000 00000000 00000000 00000000
[ 16.933913] bf80: 00000000 ba1e2c76 0000000b 0000000b 00000000 7ed85ce0 00000025 80100264
[ 16.942102] bfa0: 8296a000 00000025 7ed860ec 801000cc 00000000 0000000b 00000000 00000000
[ 16.950300] bfc0: 0000000b 00000000 7ed85ce0 00000025 7ed85c48 7ed85c4c 00000001 7ed860ec
[ 16.958490] bfe0: 00442d1c 7ed85b94 0042e318 76a64d2c 000b0010 00000101 00000000 00000000
[ 16.966703] [<80272dc8>] (__insert_inode_hash) from [<803efcf8>] (ubifs_create+0x1c4/0x210)
[ 16.975090] [<803efcf8>] (ubifs_create) from [<80266750>] (path_openat+0x4d4/0xd3c)
[ 16.982777] [<80266750>] (path_openat) from [<8026866c>] (do_filp_open+0x70/0x120)
[ 16.990370] [<8026866c>] (do_filp_open) from [<80252920>] (filp_open+0x118/0x1a4)
[ 16.997888] [<80252920>] (filp_open) from [<802c7a20>] (do_coredump+0xde4/0x11a8)
[ 17.005393] [<802c7a20>] (do_coredump) from [<801401a4>] (get_signal+0x138/0x7e4)
[ 17.012908] [<801401a4>] (get_signal) from [<8010984c>] (do_work_pending+0x110/0x594)
[ 17.020760] [<8010984c>] (do_work_pending) from [<801000cc>] (slow_work_pending+0xc/0x20)
[ 17.028958] Exception stack(0x8296bfb0 to 0x8296bff8)
[ 17.034017] bfa0: 00000000 0000000b 00000000 00000000
[ 17.042209] bfc0: 0000000b 00000000 7ed85ce0 00000025 7ed85c48 7ed85c4c 00000001 7ed860ec
[ 17.050408] bfe0: 00442d1c 7ed85b94 0042e318 76a64d2c 000b0010 00000101
[ 17.057047] Code: f57ff05b e1a00007 e7865104 e3530000 (15835004)
[ 17.063202] ---[ end trace 536dee15ce23eef6 ]---
[ 17.067843] note: systemd[257] exited with preempt_count 2
[ 17.073431] pgd = 658dd51f
[ 17.076155] [0124d494] *pgd=00000000
[ 17.079745] Internal error: Oops: 5 [#2] PREEMPT SMP ARM
[ 17.085070] Modules linked in: imx_vdoa
[ 17.088934] CPU: 0 PID: 176 Comm: systemd-journal Tainted: G D 5.10.9 #39
[ 17.097044] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 17.103589] PC is at __d_lookup_rcu+0x64/0x184
[ 17.108041] LR is at lookup_fast+0x44/0x184
[ 17.112238] pc : [<8027217c>] lr : [<80262e54>] psr: 200f0013
[ 17.118509] sp : 827c7d30 ip : 239b8cba fp : 00010001
[ 17.123738] r10: d0d0d0d0 r9 : 239b8cba r8 : 00000003
[ 17.128978] r7 : 00000003 r6 : 82a01014 r5 : 841d26e8 r4 : 0124d498
[ 17.135510] r3 : 827c7e80 r2 : 00ffffff r1 : ffffffff r0 : dac19000
[ 17.142052] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 17.149193] Control: 10c5387d Table: 12804059 DAC: 00000051
[ 17.154957] Process systemd-journal (pid: 176, stack limit = 0xa8831ba8)
[ 17.161675] Stack: (0x827c7d30 to 0x827c8000)
[ 17.166047] 7d20: 812dbe60 827c7d68 00ffffff 827ee480
[ 17.174238] 7d40: 00000001 82a01015 827c7e78 827c7d94 827c7d98 841d26e8 00000018 d0d0d0d0
[ 17.182429] 7d60: 00010001 80262e54 000000c4 ba1e2c76 82a01015 00000000 827c7e78 00000002
[ 17.190629] 7d80: 2f2f2f2f 80265b60 8429e000 00000081 00000000 827c7e78 2f2f2f2f ba1e2c76
[ 17.198828] 7da0: 827a79c0 82a01015 00000000 fefefeff 827c7e78 2f2f2f2f 00000018 80265f34
[ 17.207030] 7dc0: 00000000 61c88647 827c7e78 00000041 82a01010 80263b00 827c7e78 ba1e2c76
[ 17.215230] 7de0: 82a01010 ffffe000 82a01010 00000001 ffffff9c 827c7e78 827c7f30 827a79c0
[ 17.223430] 7e00: ffffffff 80266328 827c7f60 000000ad 7ef20674 7ef20690 7ef20710 809aa088
[ 17.231621] 7e20: 820b2000 80151f0c dad20400 00000000 00000041 827c6000 7ef2066c 00000001
[ 17.239814] 7e40: 827c7ef4 827c7ef4 00000008 ba1e2c76 00000051 82a01000 827c7f30 00000001
[ 17.248013] 7e60: ffffff9c 80100264 827c6000 00000142 ffffffff 8026866c 826943d0 841d26e8
[ 17.256212] 7e80: 239b8cba 00000003 82a01011 00000000 826943d0 841d26e8 8429e000 00000051
[ 17.264405] 7ea0: 00000002 000008de 000004ac 00000000 00000000 00000000 827c7ebc 0008b40c
[ 17.272604] 7ec0: 82a01000 80242130 827c7ed0 00000000 82a01000 ba1e2c76 827a70c0 00000015
[ 17.280802] 7ee0: 82048200 827eeb00 00000000 82a01000 00000000 00000002 ffffff9c 82a01000
[ 17.289002] 7f00: 00000000 827c7f60 ffffff9c ba1e2c76 82a01000 00000015 827c7f60 80253c98
[ 17.297201] 7f20: fffffe30 80100264 bcba855e 00000003 00020802 ba1e0000 00000006 00000100
[ 17.305400] 7f40: 00000001 ba1e2c76 80100264 ffffff9c 01c0c950 01c0c950 00000142 802540f0
[ 17.313600] 7f60: 000a0802 00000000 00000000 00000000 00000000 00000000 000a0802 00000000
[ 17.321793] 7f80: 00000000 00000000 00000000 00000000 249b5bb6 ba1e2c76 40000040 000a0802
[ 17.329991] 7fa0: 00000000 80100060 000a0802 00000000 ffffff9c 01c0c950 000a0802 00000000
[ 17.338182] 7fc0: 000a0802 00000000 01c0c950 00000142 7ef201df 7ef20370 01bfc790 ffffffff
[ 17.346380] 7fe0: 00000000 7ef20150 00000000 76ca2140 800f0010 ffffff9c 00000000 00000000
[ 17.354586] [<8027217c>] (__d_lookup_rcu) from [<80262e54>] (lookup_fast+0x44/0x184)
[ 17.362357] [<80262e54>] (lookup_fast) from [<80265b60>] (walk_component+0x38/0x1d8)
[ 17.370127] [<80265b60>] (walk_component) from [<80265f34>] (link_path_walk.part.0.constprop.0+0x234/0x358)
[ 17.379885] [<80265f34>] (link_path_walk.part.0.constprop.0) from [<80266328>] (path_openat+0xac/0xd3c)
[ 17.389296] [<80266328>] (path_openat) from [<8026866c>] (do_filp_open+0x70/0x120)
[ 17.396891] [<8026866c>] (do_filp_open) from [<80253c98>] (do_sys_openat2+0xa0/0x15c)
[ 17.404738] [<80253c98>] (do_sys_openat2) from [<802540f0>] (sys_openat+0x9c/0xc4)
[ 17.412335] [<802540f0>] (sys_openat) from [<80100060>] (ret_fast_syscall+0x0/0x54)
[ 17.420010] Exception stack(0x827c7fa8 to 0x827c7ff0)
[ 17.425079] 7fa0: 000a0802 00000000 ffffff9c 01c0c950 000a0802 00000000
[ 17.433270] 7fc0: 000a0802 00000000 01c0c950 00000142 7ef201df 7ef20370 01bfc790 ffffffff
[ 17.441460] 7fe0: 00000000 7ef20150 00000000 76ca2140
[ 17.446529] Code: ea000002 e5944000 e3540000 0a00003b (e5141004)
[ 17.452845] ---[ end trace 536dee15ce23eef7 ]---

The Ubifs seems to mount normally :

[ 3.214389] UBIFS (ubi0:0): Mounting in unauthenticated mode
[ 3.220267] UBIFS (ubi0:0): background thread "ubifs_bgt0_0" started, PID 160
[ 3.248176] UBIFS (ubi0:0): recovery needed
[ 3.344811] UBIFS (ubi0:0): recovery completed
[ 3.349514] UBIFS (ubi0:0): UBIFS: mounted UBI device 0, volume 0, name "rootfs"
[ 3.357051] UBIFS (ubi0:0): LEB size: 253952 bytes (248 KiB), min./max. I/O unit sizes: 4096 bytes/4096 bytes
[ 3.367047] UBIFS (ubi0:0): FS size: 298139648 bytes (284 MiB, 1174 LEBs), journal size 9404416 bytes (8 MiB, 38 LEBs)
[ 3.377789] UBIFS (ubi0:0): reserved for root: 0 bytes (0 KiB)
[ 3.383680] UBIFS (ubi0:0): media format: w4/r0 (latest is w5/r0), UUID 39985E51-9058-44D8-94E7-9DB9920A1755, small LPT model
[ 3.395291] VFS: Mounted root (ubifs filesystem) on device 0:15.

And the Nand is known :

[ 1.638730] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xd3
[ 1.645213] nand: Micron MT29F8G08ABACAH4
[ 1.649245] nand: 1024 MiB, SLC, erase size: 256 KiB, page size: 4096, OOB size: 224

Ubifsmount on u-boot works perfectly, I can see the file system mounted and I can load files.Here's my boot command :

bootargs=onsole=ttymxc1,115200 ubi.mtd=nandrootfs root=ubi0:rootfs rootfstype=ubifs rw mtdparts=gpmi-nand:64m(nandboot),16m(nandkernel),16m(nanddtb),16m(nandtee),-(nandrootfs)
bootcmd=nand read ${loadaddr} 0x4000000 0xc00000;nand read ${fdt_addr} 0x5000000 0x100000;if test ${tee} = yes; then nand read ${tee_addr} 0x4000000 0x400000;bootm ${tee_addr} - ${fdt_addr};else bootz ${loadaddr} - ${fdt_addr};fi

My DTS configuration :

pinctrl_gpmi_nand: gpminandgrp {
fsl,pins = <
MX6QDL_PAD_NANDF_CLE__NAND_CLE 0xb0b1
MX6QDL_PAD_NANDF_ALE__NAND_ALE 0xb0b1
MX6QDL_PAD_NANDF_CS0__NAND_CE0_B 0xb0b1
MX6QDL_PAD_NANDF_RB0__NAND_READY_B 0xb000
MX6QDL_PAD_NANDF_D0__NAND_DATA00 0xb0b1
MX6QDL_PAD_NANDF_D1__NAND_DATA01 0xb0b1
MX6QDL_PAD_NANDF_D2__NAND_DATA02 0xb0b1
MX6QDL_PAD_NANDF_D3__NAND_DATA03 0xb0b1
MX6QDL_PAD_NANDF_D4__NAND_DATA04 0xb0b1
MX6QDL_PAD_NANDF_D5__NAND_DATA05 0xb0b1
MX6QDL_PAD_NANDF_D6__NAND_DATA06 0xb0b1
MX6QDL_PAD_NANDF_D7__NAND_DATA07 0xb0b1
MX6QDL_PAD_SD4_CMD__NAND_RE_B 0xb0b1
MX6QDL_PAD_SD4_CLK__NAND_WE_B 0xb0b1
MX6QDL_PAD_NANDF_WP_B__NAND_WP_B 0xb0b1
>;
};

&gpmi {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_gpmi_nand>;
/* enable at -gpmi-weim.dts due to pin conflict */
status = "okay";
nand-on-flash-bbt;
fsl,use-minimum-ecc;
};

I flash the rootfs this way :

tftp 0x18100000 rootfs.ubifs
ubi part nandrootfs
ubi remove rootfs
ubi create rootfs
ubi write 0x18100000 rootfs 0x1322000

Boot logs and kernel .config attached.

[abrahim · Accepted solution]

Hi,

I upgraded the kernel to 5.15.11 and it worked. But I still don't know what was the problem.

Best regards,

Aïda

[Bio_TICFSL]

Hello,

It looks like you have to burn the zImage again since from the beginning has error and check you device tree the one you showed looks nice.

Regards

[abrahim]

Hello,

Can you please tell what errors ? I burned it many many times but with the same result.

Best regards,

Aïda

[Bio_TICFSL]

The error  Unable to handle kernel paging request at virtual address 0124d494 means that is no valid kernel image, you need to rebuild or burn again the Image.

Regards

[abrahim]

Hi,

I tried to rebuild with the following command line but I had the same result :

$ make -j8 ARCH=arm CROSS_COMPILE=../../host/usr/bin/arm-linux- zImage

Sometimes the error changes to :

Unable to handle kernel NULL pointer dereference at virtual address 000000a8

mtdparts from u-boot :

=> mtdparts
Device nor0 not found!
current device is invalid, skip it and check the next one

device nand0 <gpmi-nand>, # parts = 5
#: name size offset mask_flags
0: nandboot 0x04000000 0x00000000 0
1: nandkernel 0x01000000 0x04000000 0
2: nanddtb 0x01000000 0x05000000 0
3: nandtee 0x01000000 0x06000000 0
4: nandrootfs 0x39000000 0x07000000 0

active partition: nand0,0 - (nandboot) 0x04000000 @ 0x00000000

The kernel burn commands from u-boot :

mw.b 0x18100000 0xff 0x800000
tftp 0x18100000 zImage
nand erase.part nandkernel
nand write 0x18100000 nandkernel 0x800000

NB : the kernel is loaded so the magic is correct, isn't it ?