iMX7 Kernel Panic Ubuntu 18.04 On Some Boards

[Duracell]

Hi

I am booting Ubuntu 18.04 from SD card on my iMX7 board. Most modules boot perfectly. Some consistently give the below error. An help on understanding what is causing the problem would be most helpful. As it is, I am unable to understand why it happens on some boards and not others. Comparing good boot with bad boot, is seems to go wrong when the following error occurs: 

Unable to handle kernel paging request at virtual address 00400088
pgd = 80003000

BAD BOOT:

U-Boot SPL 2017.03-00940-gbb83458-dirty (Nov 13 2018 - 09:46:22)
Boot Device: SD0
Trying to boot from MMC1
Boot Device: SD0
reading u-boot.img
reading u-boot.img

U-Boot 2017.03-00940-gbb83458-dirty (Nov 13 2018 - 09:46:22 +0800)

CPU: Freescale i.MX7D rev1.2 996 MHz (running at 792 MHz)
CPU: Commercial temperature grade (0C to 95C) at 27C
Reset cause: POR
Board: PICO-IMX7D
Compatible baseboard: dwarf, hobbit, nymph, pi
I2C: ready
DRAM: 1 GiB
PMIC: PFUZE3000 DEV_ID=0x30 REV_ID=0x11
MMC: FSL_SDHC: 0, FSL_SDHC: 1
Display: MCIMX28LCD (800x480)
Video: 800x480x24
In: serial
Out: serial
Err: serial
Net: FEC0
Normal Boot
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0 is current device
reading uEnv.txt
240 bytes read in 10 ms (23.4 KiB/s)
Loaded environment from uEnv.txt
Importing environment from mmc ...
Running uenvcmd ...
reading pv-zImage
7344192 bytes read in 340 ms (20.6 MiB/s)
Booting from mmc ...
reading m4_tcm.bin
** Unable to read file m4_tcm.bin **
baseboard is pi
reading pv.dtb
40504 bytes read in 27 ms (1.4 MiB/s)
## Flattened Device Tree blob at 83000000
Booting using the fdt blob at 0x83000000
Using Device Tree in place at 83000000, end 8300ce37

Starting kernel ...

mxc_sdc_fb fb@0: NO mxc display driver found!
spi_imx 30840000.ecspi: dma setup error -19, use pio
snvs-secvio 30370000.caam-snvs: can't get snvs clock
imx-sgtl5000 sound: phandle missing or invalid
cpu cpu0: dev_pm_opp_get_opp_count: OPP table not found (-19)
Unable to handle kernel paging request at virtual address 00400088
pgd = 80003000
[00400088] *pgd=80000080004003Internal error: Oops: 206 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.9.11-1.0.0-technexion+g015d775 #1
Hardware name: Freescale i.MX7 Dual (Device Tree)
task: a8088000 task.stack: a8084000
PC is at kmem_cache_alloc+0xa0/0x1a0
LR is at __d_alloc+0x24/0x1a8
pc : [<80313904>] lr : [<8033b594>] psr: 20000013
sp : a8085e28 ip : c2d3f82a fp : 00000000
r10: 000003b9 r9 : 00400088 r8 : ffffe000
r7 : 8033b594 r6 : 024000c0 r5 : 812be5a0 r4 : a80353c0
r3 : 00000000 r2 : 2a5c7000 r1 : 000003b9 r0 : a80353c0
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 30c53c7d Table: 80003000 DAC: ba295f43
Process swapper/0 (pid: 1, stack limit = 0xa8084210)
Stack: (0xa8085e28 to 0xa8086000)
5e20: 00000000 a606e990 a8085e90 a8085e90 812033b8 a8158400
5e40: 810626e4 8033b594 00000000 a606e990 a8085e90 00000000 81221530 0000010a
5e60: 810626e4 8033b728 00000000 a606e990 a8085e90 80331e48 80e47f97 80e47f97
5e80: a606e990 80c1c2c0 81221530 80331f64 6ce7ffc3 0000000b 80e47f8c a606e990
5ea0: 812c2d64 a606e990 80e47f8c 804638e0 80c4e310 00008124 80c4e310 80463964
5ec0: 81052838 8105283c 80c4e310 80c4e310 ffffe000 81052838 8105283c 80463a2c
5ee0: 80c1c2c0 80c4e310 812cbac0 8101fa34 80c4e310 8101edc4 00000000 8101f9e0
5f00: ffffe000 802016dc 81210048 8105283c cccccccd 80382ce8 812c1200 a8142800
5f20: 00000000 8120e230 80c26ddc abfffa7c 0000010a 80247818 80f03fcc 80e04770
5f40: 80e0d710 00000007 00000007 00000000 8120e218 00000008 00000008 8126f000
5f60: 8126f000 81052838 8105283c 810626e4 00000000 81000e58 00000007 00000007
5f80: 00000000 810005a8 00000000 80ab90d0 00000000 00000000 00000000 00000000
5fa0: 00000000 80ab90d8 00000000 802076e8 00000000 00000000 00000000 00000000
5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5fe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff ffffffff
[<80313904>] (kmem_cache_alloc) from [<8033b594>] (__d_alloc+0x24/0x1a8)
[<8033b594>] (__d_alloc) from [<8033b728>] (d_alloc+0x10/0x6c)
[<8033b728>] (d_alloc) from [<80331e48>] (__lookup_hash+0x30/0x8c)
[<80331e48>] (__lookup_hash) from [<80331f64>] (lookup_one_len+0xc0/0x120)
[<80331f64>] (lookup_one_len) from [<804638e0>] (start_creating+0x6c/0xbc)
[<804638e0>] (start_creating) from [<80463964>] (__debugfs_create_file+0x34/0xcc)
[<80463964>] (__debugfs_create_file) from [<80463a2c>] (debugfs_create_file+0x30/0x38)
[<80463a2c>] (debugfs_create_file) from [<8101fa34>] (clk_debug_init+0x54/0x138)
[<8101fa34>] (clk_debug_init) from [<802016dc>] (do_one_initcall+0x44/0x170)
[<802016dc>] (do_one_initcall) from [<81000e58>] (kernel_init_freeable+0x1d0/0x260)
[<81000e58>] (kernel_init_freeable) from [<80ab90d8>] (kernel_init+0x8/0x114)
[<80ab90d8>] (kernel_init) from [<802076e8>] (ret_from_fork+0x14/0x2c)
Code: e7939002 e3590000 0a00000c e5943014 (e7990003)
---[ end trace 6a0fbf83a25deddd ]---
hub 1-1:1.0: USB hub found
hub 1-1:1.0: 7 ports detected
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

CPU0: stopping
CPU: 0 PID: 0 Comm: swapper/0 Tainted: G D 4.9.11-1.0.0-technexion+g015d775 #1
Hardware name: Freescale i.MX7 Dual (Device Tree)
[<8020ed2c>] (unwind_backtrace) from [<8020a9e8>] (show_stack+0x10/0x14)
[<8020a9e8>] (show_stack) from [<80579f58>] (dump_stack+0x78/0x8c)
[<80579f58>] (dump_stack) from [<8020dbd8>] (handle_IPI+0x198/0x1ac)
[<8020dbd8>] (handle_IPI) from [<8020140c>] (gic_handle_irq+0x88/0x8c)
[<8020140c>] (gic_handle_irq) from [<8020b478>] (__irq_svc+0x58/0x8c)
Exception stack(0x81201f18 to 0x81201f60)
1f00: 00000000 80e18d68
1f20: 00000001 2a5b9000 bb8e7e73 00000000 ab620ea8 00000000 bc4d629d 00000000
1f40: 00000000 00000050 1f400000 81201f68 8026a400 80819aac 20000013 ffffffff
[<8020b478>] (__irq_svc) from [<80819aac>] (cpuidle_enter_state+0x178/0x2ac)
[<80819aac>] (cpuidle_enter_state) from [<8026a63c>] (cpu_startup_entry+0x168/0x228)
[<8026a63c>] (cpu_startup_entry) from [<81000c7c>] (start_kernel+0x3b0/0x3bc)
[<81000c7c>] (start_kernel) from [<80008090>] (0x80008090)
---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

GOOD BOOT:

U-Boot SPL 2017.03-00940-gbb83458-dirty (Nov 13 2018 - 09:46:22)
Boot Device: SD0
Trying to boot from MMC1
Boot Device: SD0
reading u-boot.img
reading u-boot.img

U-Boot 2017.03-00940-gbb83458-dirty (Nov 13 2018 - 09:46:22 +0800)

CPU: Freescale i.MX7D rev1.2 996 MHz (running at 792 MHz)
CPU: Commercial temperature grade (0C to 95C) at 28C
Reset cause: POR
Board: PICO-IMX7D
Compatible baseboard: dwarf, hobbit, nymph, pi
I2C: ready
DRAM: 1 GiB
PMIC: PFUZE3000 DEV_ID=0x30 REV_ID=0x11
MMC: FSL_SDHC: 0, FSL_SDHC: 1
Display: MCIMX28LCD (800x480)
Video: 800x480x24
In: serial
Out: serial
Err: serial
Net: FEC0
Normal Boot
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0 is current device
reading uEnv.txt
240 bytes read in 9 ms (25.4 KiB/s)
Loaded environment from uEnv.txt
Importing environment from mmc ...
Running uenvcmd ...
reading pv-zImage
7344192 bytes read in 340 ms (20.6 MiB/s)
Booting from mmc ...
reading m4_tcm.bin
** Unable to read file m4_tcm.bin **
baseboard is pi
reading pv.dtb
40504 bytes read in 27 ms (1.4 MiB/s)
## Flattened Device Tree blob at 83000000
Booting using the fdt blob at 0x83000000
Using Device Tree in place at 83000000, end 8300ce37

Starting kernel ...

mxc_sdc_fb fb@0: NO mxc display driver found!
spi_imx 30840000.ecspi: dma setup error -19, use pio
snvs-secvio 30370000.caam-snvs: can't get snvs clock
imx-sgtl5000 sound: phandle missing or invalid
cpu cpu0: dev_pm_opp_get_opp_count: OPP table not found (-19)
cgroup: cgroup2: unknown option "nsdelegate"
[ OK ] Started Load Kernel Modules.
Mounting Kernel Configuration File System...
Starting Apply Kernel Variables...
[ OK ] Mounted Kernel Configuration File System.
[ OK ] Started Apply Kernel Variables.
[ OK ] Reached target System Initialization.
[ OK ] Started Trigger anacron every hour.
[ OK ] Reached target Paths.
[ OK ] Started Daily Cleanup of Temporary Directories.
[ OK ] Started Message of the Day.
[ OK ] Started Daily apt download activities.
[ OK ] Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System.
Starting System Logging Service...
[ OK ] Started Set the CPU Frequency Scaling governor.
Starting Dispatcher daemon for systemd-networkd...
Starting Login Service...
Starting Avahi mDNS/DNS-SD Stack...
Starting Restore /etc/resolv.conf i…fore the ppp link was shut down...
[ OK ] Started Regular background program processing daemon.
[ OK ] Started watchdog.
Starting Initialize hardware monitoring sensors...
[ OK ] Started D-Bus System Message Bus.
[ OK ] Started Login Service.
Starting WPA supplicant...
Starting Network Manager...
Starting Modem Manager...
[ OK ] Started Run anacron jobs.
[ OK ] Started Daily apt upgrade and clean activities.
[ OK ] Started Discard unused blocks once a week.
[ OK ] Reached target Timers.
Starting LSB: automatic crash report generation...
[ OK ] Started System Logging Service.
[ OK ] Started Restore /etc/resolv.conf if…before the ppp link was shut down.
[ OK ] Started Avahi mDNS/DNS-SD Stack.
[ OK ] Started Initialize hardware monitoring sensors.
[ OK ] Started WPA supplicant.
Starting Discard unused blocks...
Starting Authorization Manager...
[ OK ] Started Discard unused blocks.
[ OK ] Started Authorization Manager.
[ OK ] Started LSB: automatic crash report generation.
[ OK ] Started Modem Manager.
Starting Hostname Service...
[ OK ] Started Dispatcher daemon for systemd-networkd.
[ OK ] Started Hostname Service.
[ OK ] Started Network Manager.
Starting Network Manager Script Dispatcher Service...
Starting Network Manager Wait Online...
[ OK ] Reached target Network.
Starting OpenBSD Secure Shell server...
Starting Permit User Sessions...
[ OK ] Started Network Manager Script Dispatcher Service.
[ OK ] Started Permit User Sessions.
Starting Terminate Plymouth Boot Screen...
Starting Hold until boot process finishes up...
[ OK ] Started Terminate Plymouth Boot Screen.
[ OK ] Started Hold until boot process finishes up.
Starting Set console scheme...
[ OK ] Started Serial Getty on ttymxc4.
[ OK ] Started Set console scheme.
[ OK ] Created slice system-getty.slice.
[ OK ] Started Getty on tty1.
[ OK ] Reached target Login Prompts.
[ OK ] Started OpenBSD Secure Shell server.

etc

[Bio_TICFSL]

Hello Duracell,

You need to erase and burn again the program on the i.mx, this look like a bad installation or something is bad on your dtb.

Regards

[Duracell]

Hi

Can you please elaborate. I don't understand how it can be a bad image. I have a baseboard that I attached the iMX7 modules to. Some modules execute perfectly repeatedly while other fail repeatedly. The image is on an SD card on the baseboard. If the image were bad, I would have expected all iMX7 modules to fail consistently.

Thanks