Consistent Kernel Panic in arch_idle

road-iq — Fri, 22 May 2015 21:48:57 GMT

I have an iMX6Q platform that is derived from the iMX6 automotive reference design. It runs Linux 3.0.35 (with some caching patches) built with Yocto.

The CPU runs at 800Mhz. Memory bus at 528Mhz. VPU at 326Mhz. We are streaming continuous video from a TW6869 through the IPU to the VPU and finally out the network (automotive surround view camera for large vehicles). We are consistently getting an kernel panic in arch_idle when we let it run over time, sometimes minutes, sometimes hours.

I see that others have had issues with the GPT and arch_idle but the backtraces and symptoms do not match what we are seeing.

I'm looking for clues as to what we could be doing wrong here.

I am getting the following dump after the crash:

] PREEMPT SMP

[ 5045.584304] Modules linked in: iptable_nat nf_nat iptable_filter ip_tables unifi_sdio option usb_wwan usbserial tw68v

[ 5045.595067] CPU: 0 Not tainted (3.0.35-2508-g54750ff-1.1.0_beta #1)

[ 5045.601695] PC is at cmd_line+0x4/0x400

[ 5045.605543] LR is at arch_idle+0xf8/0x130

[ 5045.609562] pc : [<8002a0a8>] lr : [<80044f6c>] psr: 40070013

[ 5045.609567] sp : 80557fc8 ip : 00000000 fp : 00000000

[ 5045.621061] r10: 00000000 r9 : 412fc09a r8 : 80556000 setting 56[ 5045.626291] r7 : 8056f9dc r6 : 8040cf9c r5 : 80556000 r4 : 8056ba20

[ 5045.634727] r3 : 80556000 r2 : 00000000 r1 : 80070093 r0 : 00000000

[ 5045.641268] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel

[ 5045.648590] Control: 10c53c7d Table: 481ec04a DAC: 00000015

[ 5045.654351] Process swapper (pid: 0, stack limit = 0x805562f0)

[ 5045.660197] Stack: (0x80557fc8 to 0x80558000)

[ 5045.664567] 7fc0: 8c005100 8000891c 800083ac 00000000 00000000 8002a0testing 56[ 5045.672756] 7fe0: 10c53c7d 8056ba8c 8002a0a0 8056f9d4 1000406a 10008040 00000000 00000000

[ 5045.682876] [<8002a0a8>] (cmd_line+0x4/0x400) from [<800083ac>] (unknown_bootoption+0x0/0x218)

[ 5045.691508] [<800083ac>] (unknown_bootoption+0x0/0x218) from [<1000406a>] (0x1000406a)

[ 5045.699438] Code: 00000000 623f3f6c 10000100 736e6f63 (3d656c6f)

[ 5045.705622] ---[ end trace 0a173dd945b3c51f ]---

[ 5045.710256] Kernel panic - not syncing: Attempted to kill the idle task!

[ 5045.717055] [<8003cd68>] (unwind_backtrace+0x0/0xec) from [<80406084>] (panic+0x80/0x19setting 57[ 5045.725317] [<80406084>] (panic+0x80/0x194) from [<8006bd38>] (do_exit+0x5b0/0x708)

[ 5045.734929] [<8006bd38>] (do_exit+0x5b0/0x708) from [<80039834>] (die+0x264/0x2d0)

[ 5045.742539] [<80039834>] (die+0x264/0x2d0) from [<80030154>] (do_undefinstr+0x150/0x15c)

[ 5045.750836] [<80030154>] (do_undefinstr+0x150/0x15c) from [<80035b04>] (__und_svc+0x44/testing 57[ 5045.759411] Exception stack(0x80557f80 to 0x80557fc8)

[ 5045.766316] 7f80: 00000000 80070093 00000000 80556000 8056ba20 80556000 8040cf9c 8056f9dc

[ 5045.774570] 7fa0: 80556000 412fc09a 00000000 00000000 00000000 80557fc8 80044f6c 8002a0a8

[ 5045.782785] 7fc0: 40070013 ffffffff

[ 5045.786350] [<80035b04>] (__und_svc+0x44/0x60) from [<8002a0a8>] (cmd_line+0x4/0x400)

[ 5045.794243] [<8002a0a8>] (cmd_line+0x4/0x400) from [<800083ac>] (unknown_bootoption+0x0setting 58[ 5045.802907] [<800083ac>] (unknown_bootoption+0x0/0x218) from [<1000406a>] (0x1000406a)

[ 5045.812783] CPU1: stopping

[ 5045.815529] [<8003cd68>] (unwind_backtrace+0x0/0xec) from [<80030338>] (do_IPI+0x144/0x154)

[ 5045.823895] [<80030338>] (do_IPI+0x144/0x154) from [<80035a0c>] (__irq_svc+0x4c/0xe8)

[ 5045.831729] Exception stack(0xbffadf90 to 0xbffadfd8)

[ 5045.836788] df80: 805dea40 80070093 00000001 00000000

[ 5045.844974] dfa0: 8059b564 8056ba6c 8040cf9c 8056f9dc bffac000 412fc09a 00000000 00000000

[ 5045.853158] dfc0: 00000000 bffadfd8 80044f6c 80036b98 40070013 ffffffff

[ 5045.859789] [<80035a0c>] (__irq_svc+0x4c/0xe8) from [<80036b98>] (default_idle+0x24/0x28)

[ 5045.867981] [<80036b98>] (default_idle+0x24/0x28) from [<80036d0c>] (cpu_idle+0xbc/0x120)

[ 5045.876172] [<80036d0c>] (cpu_idle+0xbc/0x120) from [<10402bf4>] (0x10402bf4)

[ 5045.883313] CPU2: stopping

[ 5045.886039] [<8003cd68>] (unwind_backtrace+0x0/0xec) from [<80030338>] (do_IPI+0x144/0x154)

[ 5045.894404] [<80030338>] (do_IPI+0x144/0x154) from [<80035a0c>] (__irq_svc+0x4c/0xe8)

[ 5045.902238] Exception stack(0xbffb5f90 to 0xbffb5fd8)

[ 5045.907295] 5f80: 805dea40 800f0093 00000001 00000000

[ 5045.915484] 5fa0: 8059b564 8056ba6c 8040cf9c 8056f9dc bffb4000 412fc09a 00000000 00000000

[ 5045.923668] 5fc0: 00000000 bffb5fd8 80044f6c 80036b98 400f0013 ffffffff

[ 5045.930295] [<80035a0c>] (__irq_svc+0x4c/0xe8) from [<80036b98>] (default_idle+0x24/0x28)

[ 5045.938485] [<80036b98>] (default_idle+0x24/0x28) from [<80036d0c>] (cpu_idle+0xbc/0x120)

[ 5045.946674] [<80036d0c>] (cpu_idle+0xbc/0x120) from [<10402bf4>] (0x10402bf4)

[ 5045.953815] CPU3: stopping

[ 5045.956542] [<8003cd68>] (unwind_backtrace+0x0/0xec) from [<80030338>] (do_IPI+0x144/0x154)

[ 5045.964907] [<80030338>] (do_IPI+0x144/0x154) from [<80035ca8>] (__irq_usr+0x48/0xe0)

[ 5045.972741] Exception stack(0xbadebfb0 to 0xbadebff8)

[ 5045.977798] bfa0: 00000000 00000000 00000000 0007edbd

[ 5045.985984] bfc0: 2b401ef4 0000003a 00013334 2b206804 2ad07000 ffffffff 0013fe01 2ad07000

[ 5045.994168] bfe0: ffffffff 7ee5ebe0 2af026f0 00009c3c 40030010 ffffffff

Re: Consistent Kernel Panic in arch_idle

igorpadykov — Mon, 25 May 2015 09:34:56 GMT

Hi Mark

one can try kernel boot parameter enable_wait_mode=off

and apply Linux 3.0.35 stability pacthes below

IMX6LXPATCHRN i.MX 6Quad, i.MX 6Dual, i.MX 6DualLite, i.MX 6Solo and i.MX 6Sololite Linux

BSP Documentation. Patch Release Notes

Re: Re: IMX6Q system hang-up problem / linux kernel(3.0.35)

note, last BSP L3.14.28_1.0.0_iMX6QDLS_BUNDLE

includes them all so one can cosider to migrate to it

Also one can try to tweak memory

Kernel panic - not syncing: Attempted to kill init!

https://community.freescale.com/thread/323018

Best regards

igor

Re: Consistent Kernel Panic in arch_idle

road-iq — Thu, 28 May 2015 22:05:13 GMT

We are applying some of these patches. The key to avoiding our crash we found was to set enable_wait_mode=off in uboot.

This basically avoided the bits of code in arch_idle that could cause the crash.

From the links you sent it appears that a roll up of all the 3.0.35 patchs is now called 3.0.101. Is that correct?

Re: Consistent Kernel Panic in arch_idle

igorpadykov — Sat, 30 May 2015 14:41:32 GMT

yes, it is correct.

~igor

topic Re: Consistent Kernel Panic in arch_idle in i.MX Processors

Consistent Kernel Panic in arch_idle

Re: Consistent Kernel Panic in arch_idle

Re: Consistent Kernel Panic in arch_idle

Re: Consistent Kernel Panic in arch_idle