AnsweredAssumed Answered

Reason for kernel panic ? Unable to handle kernel paging request at virtual address 7e263df8 / Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: 80193638

Question asked by Peter Amond on Dec 17, 2017
Latest reply on Dec 18, 2017 by Peter Amond

Dear All,

 

This is regarding imx6q, 2GB Alliance Memory (4Gb x 4, 4 RAM chips) custom hardware design which is most similar to nitrogen6_max. I have manufactured 50 units of this design and I could get work only 10 boards out of  50 without any kernel panics.

 

I did memory calibration properly and did stress test about 4 hours and it passed without any conflict.

 

Other boards get kernel panics randomly. Here I have attached kernel log. Is this a memory problem ? When I do the program counter decoding using addr2line it shows >>Problem related to scheduling.

 

Any idea of this ?

 

 

Regards.

 

Kernel Log

 

Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: 80193638

CPU2: stopping
CPU: 2 PID: 0 Comm: swapper/2 Tainted: G O 4.1.15-1.0.0_ga+yocto+g76c24a0 #8
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<8010b528>] (dump_backtrace) from [<8010b748>] (show_stack+0x18/0x1c)
r7:00000000 r6:80d1c2e0 r5:80d1c2e0 r4:00000000
[<8010b730>] (show_stack) from [<80862be8>] (dump_stack+0x8c/0xcc)
[<80862b5c>] (dump_stack) from [<8010e72c>] (handle_IPI+0x180/0x194)
r7:00000000 r6:00000005 r5:00000002 r4:80c60c04
[<8010e5ac>] (handle_IPI) from [<801014a4>] (gic_handle_irq+0x60/0x64)
r9:0000002e r8:f6b72f95 r7:f4a00100 r6:ce0a7f08 r5:80d02888 r4:f4a0010c
[<80101444>] (gic_handle_irq) from [<8010c340>] (__irq_svc+0x40/0x74)
Exception stack(0xce0a7f08 to 0xce0a7f50)
7f00: 00000000 d0f23440 dc8ba30f dc8ba30f 80d02508 d0f1fed0
7f20: f6ab55bb 0000002e f6b72f95 0000002e 00000001 ce0a7f94 ce0a7ef0 ce0a7f50
7f40: 808683a4 8063c8cc 60010013 ffffffff
r7:ce0a7f3c r6:ffffffff r5:60010013 r4:8063c8cc
[<8063c7dc>] (cpuidle_enter_state) from [<8063ca68>] (cpuidle_enter+0x1c/0x20)
r10:00000001 r9:ce0a7fa8 r8:80d07ad0 r7:d0f1fed0 r6:809028c0 r5:80d025e8
r4:ce0a6000
[<8063ca4c>] (cpuidle_enter) from [<80168280>] (cpu_startup_entry+0x208/0x330)
[<80168078>] (cpu_startup_entry) from [<8010e348>] (secondary_start_kernel+0x150/0x15c)
r7:80d51308
[<8010e1f8>] (secondary_start_kernel) from [<1010154c>] (0x1010154c)
r5:00000015 r4:5e08806a
CPU1: stopping
CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 4.1.15-1.0.0_ga+yocto+g76c24a0 #8
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<8010b528>] (dump_backtrace) from [<8010b748>] (show_stack+0x18/0x1c)
r7:00000000 r6:80d1c2e0 r5:80d1c2e0 r4:00000000
[<8010b730>] (show_stack) from [<80862be8>] (dump_stack+0x8c/0xcc)
[<80862b5c>] (dump_stack) from [<8010e72c>] (handle_IPI+0x180/0x194)
r7:00000000 r6:00000005 r5:00000001 r4:80c60c04
[<8010e5ac>] (handle_IPI) from [<801014a4>] (gic_handle_irq+0x60/0x64)
r9:0000002e r8:f6b730e3 r7:f4a00100 r6:ce0a5f08 r5:80d02888 r4:f4a0010c
[<80101444>] (gic_handle_irq) from [<8010c340>] (__irq_svc+0x40/0x74)
Exception stack(0xce0a5f08 to 0xce0a5f50)
5f00: 00000000 d0f17440 dc8ba30f dc8ba30f 80d02508 d0f13ed0
5f20: f631f76b 0000002e f6b730e3 0000002e 00000001 ce0a5f94 ce0a5ef0 ce0a5f50
5f40: 808683a4 8063c8cc 60030013 ffffffff
r7:ce0a5f3c r6:ffffffff r5:60030013 r4:8063c8cc
[<8063c7dc>] (cpuidle_enter_state) from [<8063ca68>] (cpuidle_enter+0x1c/0x20)
r10:00000001 r9:ce0a5fa8 r8:80d07ad0 r7:d0f13ed0 r6:809028c0 r5:80d025e8
r4:ce0a4000
[<8063ca4c>] (cpuidle_enter) from [<80168280>] (cpu_startup_entry+0x208/0x330)
[<80168078>] (cpu_startup_entry) from [<8010e348>] (secondary_start_kernel+0x150/0x15c)
r7:80d51308
[<8010e1f8>] (secondary_start_kernel) from [<1010154c>] (0x1010154c)
r5:00000015 r4:5e08806a
CPU3: stopping
CPU: 3 PID: 0 Comm: swapper/3 Tainted: G O 4.1.15-1.0.0_ga+yocto+g76c24a0 #8
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<8010b528>] (dump_backtrace) from [<8010b748>] (show_stack+0x18/0x1c)
r7:00000000 r6:80d1c2e0 r5:80d1c2e0 r4:00000000
[<8010b730>] (show_stack) from [<80862be8>] (dump_stack+0x8c/0xcc)
[<80862b5c>] (dump_stack) from [<8010e72c>] (handle_IPI+0x180/0x194)
r7:00000000 r6:00000005 r5:00000003 r4:80c60c04
[<8010e5ac>] (handle_IPI) from [<801014a4>] (gic_handle_irq+0x60/0x64)
r9:0000002e r8:f6b730e3 r7:f4a00100 r6:ce0a9f08 r5:80d02888 r4:f4a0010c
[<80101444>] (gic_handle_irq) from [<8010c340>] (__irq_svc+0x40/0x74)
Exception stack(0xce0a9f08 to 0xce0a9f50)
9f00: 00000000 80d97c28 dc8ba30f dc8ba30f 80d02508 d0f2bed0
9f20: f6445fa8 0000002e f6b730e3 0000002e 00000001 ce0a9f94 ce0a9ef0 ce0a9f50
9f40: 808683a4 8063c8cc 600e0013 ffffffff
r7:ce0a9f3c r6:ffffffff r5:600e0013 r4:8063c8cc
[<8063c7dc>] (cpuidle_enter_state) from [<8063ca68>] (cpuidle_enter+0x1c/0x20)
r10:00000001 r9:ce0a9fa8 r8:80d07ad0 r7:d0f2bed0 r6:809028c0 r5:80d025e8
r4:ce0a8000
[<8063ca4c>] (cpuidle_enter) from [<80168280>] (cpu_startup_entry+0x208/0x330)
[<80168078>] (cpu_startup_entry) from [<8010e348>] (secondary_start_kernel+0x150/0x15c)
r7:80d51308
[<8010e1f8>] (secondary_start_kernel) from [<1010154c>] (0x1010154c)
r5:00000015 r4:5e08806a
---[ end Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: 80193638

------------[ cut here ]------------
WARNING: CPU: 0 PID: 0 at /home/tengri/Desktop/YOCTO_wayland/core-image-minimal/tmp/work-shared/nitrogen6x/kernel-source/kernel/time/tick-broadcast.c:621 tick_handle_oneshot_broadcast+0x1f4/0x218()
Modules linked in: brcmfmac brcmutil cfg80211 usb_f_ecm g_ether usb_f_rndis libcomposite u_ether tw6869 videobuf2_dma_contig videobuf2_memops mxc_v4l2_capture ipu_bg_overlay_sdc ipu_still ipu_prp_enc ipu_csi_enc ipu_fg_overlay_sdc adv7610_video v4l2_int_device galcore(O)
CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.1.15-1.0.0_ga+yocto+g76c24a0 #8
Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
Backtrace:
[<8010b528>] (dump_backtrace) from [<8010b748>] (show_stack+0x18/0x1c)
r7:8019164c r6:80d1c2e0 r5:80d1c2e0 r4:00000000
[<8010b730>] (show_stack) from [<80862be8>] (dump_stack+0x8c/0xcc)
[<80862b5c>] (dump_stack) from [<80130148>] (warn_slowpath_common+0x88/0xb8)
r7:8019164c r6:0000026d r5:00000009 r4:00000000
[<801300c0>] (warn_slowpath_common) from [<80130264>] (warn_slowpath_null+0x24/0x2c)
r8:ffffffff r7:80d02650 r6:00000004 r5:80c60648 r4:80d4e9ee
[<80130240>] (warn_slowpath_null) from [<8019164c>] (tick_handle_oneshot_broadcast+0x1f4/0x218)
[<80191458>] (tick_handle_oneshot_broadcast) from [<8011b084>] (mxc_timer_interrupt+0x34/0x3c)
r10:80d4e9ab r9:ce003cc0 r8:00000011 r7:00000000 r6:00000000 r5:ce003d20
r4:00000001
[<8011b050>] (mxc_timer_interrupt) from [<80171a1c>] (handle_irq_event_percpu+0x80/0x13c)
r5:ce003d20 r4:80d07900
[<8017199c>] (handle_irq_event_percpu) from [<80171b1c>] (handle_irq_event+0x44/0x64)
r10:80d01d38 r9:ce008000 r8:00000001 r7:80d01e48 r6:80d07900 r5:ce003d20
r4:ce003cc0
[<80171ad8>] (handle_irq_event) from [<80174cf0>] (handle_fasteoi_irq+0xe8/0x198)
r7:80d01e48 r6:80d09170 r5:ce003d20 r4:ce003cc0
[<80174c08>] (handle_fasteoi_irq) from [<80171014>] (generic_handle_irq+0x34/0x44)
r7:80d01e48 r6:00000011 r5:00000000 r4:00000011
[<80170fe0>] (generic_handle_irq) from [<8017130c>] (__handle_domain_irq+0x84/0xf4)
r5:00000000 r4:80c60c04
[<80171288>] (__handle_domain_irq) from [<80101470>] (gic_handle_irq+0x2c/0x64)
r10:80d01e48 r9:ce008000 r8:00000001 r7:f4a00100 r6:80d01d38 r5:80d02888
r4:f4a0010c r3:80d01d38
[<80101444>] (gic_handle_irq) from [<8010c340>] (__irq_svc+0x40/0x74)
Exception stack(0x80d01d38 to 0x80d01d80)
1d20: 00000000 80a9c1c0
1d40: 80d539c0 00000000 00000082 00000000 00000010 00000000 00000001 ce008000
1d60: 80d01e48 80d01ddc 80d01d70 80d01d80 80419e58 8013375c 200e0113 ffffffff
r7:80d01d6c r6:ffffffff r5:200e0113 r4:8013375c
[<801336a4>] (__do_softirq) from [<80133c04>] (irq_exit+0xc8/0x104)
r10:80d01e48 r9:ce008000 r8:00000001 r7:00000000 r6:00000010 r5:00000000
r4:80c60c04
[<80133b3c>] (irq_exit) from [<80171310>] (__handle_domain_irq+0x88/0xf4)
r5:00000000 r4:80c60c04
[<80171288>] (__handle_domain_irq) from [<80101470>] (gic_handle_irq+0x2c/0x64)
r10:d0f0b440 r9:ffffd99a r8:80d02509 r7:f4a00100 r6:80d01e48 r5:80d02888
r4:f4a0010c r3:80d01e48
[<80101444>] (gic_handle_irq) from [<8010c340>] (__irq_svc+0x40/0x74)
Exception stack(0x80d01e48 to 0x80d01e90)
1e40: 0000005d 00000000 dc8ba30f dc8ba30f 00000000 80a953b8
1e60: 00000000 80d53528 80d02509 ffffd99a d0f0b440 80d01eb4 80d01de0 80d01e90
1e80: 80170054 80861ab8 600e0013 ffffffff
r7:80d01e7c r6:ffffffff r5:600e0013 r4:80861ab8
[<80861928>] (panic) from [<8012ff4c>] (print_tainted+0x0/0xb8)
r3:d6dc8ba3 r2:dc8ba30f r1:80193638 r0:80a953b8
r7:0000002f
[<8012ff30>] (__stack_chk_fail) from [<80193638>] (__tick_nohz_idle_enter+0x4c4/0x4cc)
[<80193174>] (__tick_nohz_idle_enter) from [<8019393c>] (tick_nohz_idle_enter+0x48/0x80)
r10:00000001 r9:80d01f58 r8:00000001 r7:d0f07ed0 r6:809028c0 r5:80d025e8
r4:80c60650
[<801938f4>] (tick_nohz_idle_enter) from [<801680f4>] (cpu_startup_entry+0x7c/0x330)
r5:80d025e8 r4:80d00000
[<80168078>] (cpu_startup_entry) from [<80860b48>] (rest_init+0x90/0x94)
r7:80d02500
[<80860ab8>] (rest_init) from [<80c00d90>] (start_kernel+0x400/0x420)
r5:80d51000 r4:80d5104c
[<80c00990>] (start_kernel) from [<1000807c>] (0x1000807c)
---[ end trace 96a3428ae6d0ad12 ]---

Outcomes