i.MX8MQ is having kernel panic

harshitshah — Tue, 21 Dec 2021 06:54:28 GMT

Hi Team,

We are using the i.MX8MQ based board, during boot-up we have found that it is giving below kernel crash.

We are using i.MX8MQ processor custom board on L5.4.3_1.0.0 version. (We are not planning to modify the same if this is not a known issue).

=============================================================================== [2021-12-16 10:25:01] [ 69.370450] Unable to handle kernel paging request at virtual address ffff8000eabac000 [2021-12-16 10:25:01] [ 69.378373] Mem abort info: [2021-12-16 10:25:01] [ 69.381164] ESR = 0x86000005 [2021-12-16 10:25:01] [ 69.384216] EC = 0x21: IABT (current EL), IL = 32 bits [2021-12-16 10:25:01] [ 69.389525] SET = 0, FnV = 0 [2021-12-16 10:25:01] [ 69.392576] EA = 0, S1PTW = 0 [2021-12-16 10:25:01] [ 69.395716] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000041992000 [2021-12-16 10:25:01] [ 69.402414] [ffff8000eabac000] pgd=000000013ffff003, pud=0000000000000000 [2021-12-16 10:25:01] [ 69.409203] Internal error: Oops: 86000005 [#1] PREEMPT SMP [2021-12-16 10:25:01] [ 69.414773] Modules linked in: [2021-12-16 10:25:01] [ 69.417830] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.4.3-lts-lf-5.4.y+gf811858 #1 [2021-12-16 10:25:01] [ 69.425569] Hardware name: i.MX8MQ Processor (DT) [2021-12-16 10:25:01] [ 69.431660] pstate: 80000085 (Nzcv daIf -PAN -UAO) [2021-12-16 10:25:01] [ 69.436453] pc : 0xffff8000eabac000 [2021-12-16 10:25:01] [ 69.439946] lr : ktime_get+0x40/0x98 [2021-12-16 10:25:01] [ 69.443517] sp : ffff80001511bef0 [2021-12-16 10:25:01] [ 69.446829] x29: ffff80001511bef0 x28: 0000000000000000 [2021-12-16 10:25:01] [ 69.452138] x27: 0000000000000000 x26: 0000000000000000 [2021-12-16 10:25:01] [ 69.457448] x25: 0000000000000000 x24: 0000000000000000 [2021-12-16 10:25:01] [ 69.462758] x23: ffff0000f80c1c00 x22: 00000000000060ba [2021-12-16 10:25:01] [ 69.468068] x21: 0000000fed0e376b x20: ffff800014f11b00 [2021-12-16 10:25:01] [ 69.473377] x19: ffff0000ff791d70 x18: ffff0000f3234600 [2021-12-16 10:25:01] [ 69.478687] x17: 0000000000000000 x16: 0000000000000000 [2021-12-16 10:25:01] [ 69.483997] x15: ffff0000f20b3f80 x14: 0000000040000010 [2021-12-16 10:25:01] [ 69.489307] x13: ffff0000ff750000 x12: 0000000000000001 [2021-12-16 10:25:01] [ 69.494616] x11: 0000000000000000 x10: 00000000000009c0 [2021-12-16 10:25:01] [ 69.499926] x9 : ffff80001511be90 x8 : ffff0000f80c2620 [2021-12-16 10:25:01] [ 69.505236] x7 : 0000000000000000 x6 : ffff0000f83a46f0 [2021-12-16 10:25:01] [ 69.510546] x5 : 0000000000000007 x4 : 0000000000000000 [2021-12-16 10:25:01] [ 69.515855] x3 : ffff800010c7d140 x2 : 0000000000000000 [2021-12-16 10:25:01] [ 69.521165] x1 : ffff8000eabac000 x0 : ffff800014e9ff28 [2021-12-16 10:25:01] [ 69.526476] Call trace: [2021-12-16 10:25:01] [ 69.528920] 0xffff8000eabac000 [2021-12-16 10:25:01] [ 69.532063] tick_nohz_idle_enter+0x3c/0x70 [2021-12-16 10:25:01] [ 69.536246] do_idle+0x34/0x280 [2021-12-16 10:25:01] [ 69.539386] cpu_startup_entry+0x24/0x40 [2021-12-16 10:25:01] [ 69.543310] secondary_start_kernel+0x154/0x190 [2021-12-16 10:25:01] [ 69.547844] Code: bad PC value [2021-12-16 10:25:01] [ 69.550903] ---[ end trace bc3e5a44512dce5e ]--- [2021-12-16 10:25:01] [ 69.555520] Kernel panic - not syncing: Attempted to kill the idle task! [2021-12-16 10:25:01] [ 69.562219] SMP: stopping secondary CPUs [2021-12-16 10:25:01] [ 69.566146] Kernel Offset: disabled [2021-12-16 10:25:01] [ 69.569633] CPU features: 0x0002,2000200c [2021-12-16 10:25:01] [ 69.573639] Memory Limit: none [2021-12-16 10:25:01] [ 69.576695] Rebooting in 10 seconds.. [2021-12-16 10:25:13] =============================================================================== [2021-12-16 10:26:39] [ 79.198401] Unable to handle kernel paging request at virtual address ffff80001582c570 [2021-12-16 10:26:39] [ 79.206324] Mem abort info: [2021-12-16 10:26:39] [ 79.209115] ESR = 0x96000007 [2021-12-16 10:26:39] [ 79.212168] EC = 0x25: DABT (current EL), IL = 32 bits [2021-12-16 10:26:39] [ 79.217476] SET = 0, FnV = 0 [2021-12-16 10:26:39] [ 79.220528] EA = 0, S1PTW = 0 [2021-12-16 10:26:39] [ 79.223665] Data abort info: [2021-12-16 10:26:39] [ 79.226542] ISV = 0, ISS = 0x00000007 [2021-12-16 10:26:39] [ 79.230374] CM = 0, WnR = 0 [2021-12-16 10:26:39] [ 79.233341] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000041992000 [2021-12-16 10:26:39] [ 79.240038] [ffff80001582c570] pgd=000000013ffff003, pud=000000013fffe003, pmd=0000000132307003, pte=0000000000000000 [2021-12-16 10:26:39] [ 79.250649] Internal error: Oops: 96000007 [#1] PREEMPT SMP [2021-12-16 10:26:39] [ 79.256219] Modules linked in: [2021-12-16 10:26:39] [ 79.259276] CPU: 0 PID: 309 Comm: 30902000.jr-eng Not tainted 5.4.3-lts-lf-5.4.y+gf811858 #1 [2021-12-16 10:26:39] [ 79.267709] Hardware name: i.MX8MQ Processor (DT) [2021-12-16 10:26:39] [ 79.273800] pstate: 80000085 (Nzcv daIf -PAN -UAO) [2021-12-16 10:26:39] [ 79.278597] pc : pick_next_task_idle+0x30/0x48 [2021-12-16 10:26:39] [ 79.283039] lr : pick_next_task_idle+0x2c/0x48 [2021-12-16 10:26:39] [ 79.287479] sp : ffff80001582bd60 [2021-12-16 10:26:39] [ 79.290791] x29: ffff80001582bd60 x28: 0000000000000000 [2021-12-16 10:26:39] [ 79.296101] x27: ffff80001582bde0 x26: ffff0000f8372180 [2021-12-16 10:26:39] [ 79.301410] x25: ffff800010fa5ff0 x24: ffff800010f6fc98 [2021-12-16 10:26:39] [ 79.306720] x23: ffff0000f8371c00 x22: ffff800014bdd018 [2021-12-16 10:26:39] [ 79.312030] x21: 0000000000000000 x20: ffff0000f8371c00 [2021-12-16 10:26:39] [ 79.317340] x19: ffff800014dd1e40 x18: 0000000000000358 [2021-12-16 10:26:39] [ 79.322649] x17: 0000000000000000 x16: 0000000000000000 [2021-12-16 10:26:39] [ 79.327959] x15: 0000000000000000 x14: 0000000040000010 [2021-12-16 10:26:39] [ 79.333268] x13: 0000000000000000 x12: 0000000000004722 [2021-12-16 10:26:39] [ 79.338578] x11: 00000000000001d7 x10: 00000000011ce9bc [2021-12-16 10:26:39] [ 79.343888] x9 : 0000000000004722 x8 : 0000000000000000 [2021-12-16 10:26:39] [ 79.349197] x7 : 0000000000000000 x6 : ffff0000f8371cf0 [2021-12-16 10:26:39] [ 79.354507] x5 : 00000000ffffffff x4 : 0000000000000000 [2021-12-16 10:26:39] [ 79.359816] x3 : ffff800010114bd8 x2 : ffff800010119540 [2021-12-16 10:26:39] [ 79.365126] x1 : 0000000000000000 x0 : ffff800014dd1e40 [2021-12-16 10:26:39] [ 79.370436] Call trace: [2021-12-16 10:26:39] [ 79.372882] pick_next_task_idle+0x30/0x48 [2021-12-16 10:26:39] [ 79.376978] __schedule+0x3d4/0x560 [2021-12-16 10:26:39] [ 79.380464] schedule+0x40/0xe0 [2021-12-16 10:26:39] [ 79.383607] kthread_worker_fn+0xf8/0x188 [2021-12-16 10:26:39] [ 79.387614] kthread+0xf0/0x120 [2021-12-16 10:26:39] [ 79.390756] ret_from_fork+0x10/0x18 [2021-12-16 10:26:39] [ 79.394334] Code: f9404022 f9401c42 d63f0040 aa1303e0 (f9400bf3) [2021-12-16 10:26:39] [ 79.400426] ---[ end trace 5d29fc571c53c860 ]--- [2021-12-16 10:26:39] [ 79.405042] note: 30902000.jr-eng[309] exited with preempt_count 2

What can be the reason for the same? We have followed the hardware guidelines mentioned in the document and performed the DDR calibration.

Regards.

Re: i.MX8MQ is having kernel panic

jimmychan — Thu, 23 Dec 2021 03:01:02 GMT

How many boards have this issue? Is the crash randomly? or crash at the same place during boot? Is the DDR stress test passed?

Re: i.MX8MQ is having kernel panic

harshitshah — Thu, 23 Dec 2021 06:21:27 GMT

Hi @jimmychan ,

There are 3 boards that have this issue out of 15. This crash is happening at the same boot-up, and we are not getting the fs prompt.

Yes, DDR stress test in the overnight case is passing.

Regards.

Re: i.MX8MQ is having kernel panic

jimmychan — Thu, 23 Dec 2021 07:05:45 GMT

Could you provide the full boot up log of the good board and the failed board?

i.MX ProcessorsのトピックRe: i.MX8MQ is having kernel panic

i.MX8MQ is having kernel panic

Re: i.MX8MQ is having kernel panic

Re: i.MX8MQ is having kernel panic

Re: i.MX8MQ is having kernel panic