RCU stall on kernel 4.4.240

ivang — Tue, 07 Jun 2022 10:24:41 GMT

Hi,

I am getting occasionally kernels hanging (sometimes hours after boot) with the following output occurring and that just goes on until i reboot the machine:

INFO: rcu_sched self-detected stall on CPU 1-...: (52300 ticks this GP) idle=439/140000000000001/0 softirq=806264/806264 fqs=52291 INFO: rcu_sched detected stalls on CPUs/tasks: 1-...: (52300 ticks this GP) idle=439/140000000000001/0 softirq=806264/806264 fqs=52291 (detected by 0, t=52542 jiffies, g=463552, c=463551, q=391502) Task dump for CPU 1: WQ#1(System) R running 4640 28173 28140 0x00000002 Backtrace: [<beb5bd7c>] (0xbeb5bd7c) from [<beb5bdac>] (0xbeb5bdac) Backtrace aborted due to bad frame pointer <8007759c> (t=52545 jiffies g=463552 c=463551 q=391595) Task dump for CPU 1: WQ#1(System) R running 4640 28173 28140 0x00000002 Backtrace: [<80014a38>] (dump_backtrace) from [<80014c80>] (show_stack+0x20/0x24) r7:807b26c0 r6:60010193 r5:807b26c0 r4:bcfbae00 [<80014c60>] (show_stack) from [<80057204>] (sched_show_task+0xbc/0x118) [<80057148>] (sched_show_task) from [<80059354>] (dump_cpu_task+0x3c/0x4c) r5:807b26c0 r4:00000001 [<80059318>] (dump_cpu_task) from [<80082344>] (rcu_dump_cpu_stacks+0x94/0xd8) r5:807b26c0 r4:00000001 [<800822b0>] (rcu_dump_cpu_stacks) from [<80086698>] (rcu_check_callbacks+0x548/0x860) r9:bf7d3100 r8:807b26c0 r7:807a460c r6:3f033000 r5:807a4584 r4:807a0100 [<80086150>] (rcu_check_callbacks) from [<8008aca4>] (update_process_times+0x48/0xa8) r10:8009d204 r9:bf7cfc98 r8:805989c4 r7:00002fd9 r6:bcfbae00 r5:ffffe000 r4:00000000 [<8008ac5c>] (update_process_times) from [<8009d200>] (tick_sched_handle+0x58/0x5c) r7:00002fd9 r6:23800e91 r5:beb5bde8 r4:bf7cfdb8 [<8009d1a8>] (tick_sched_handle) from [<8009d258>] (tick_sched_timer+0x54/0x9c) [<8009d204>] (tick_sched_timer) from [<8008b9fc>] (__hrtimer_run_queues+0x160/0x324) r7:bf7cfc00 r6:beb5a000 r5:00000000 r4:bf7cfdb8 [<8008b89c>] (__hrtimer_run_queues) from [<8008c2e8>] (hrtimer_interrupt+0xc8/0x22c) r10:bf7cfc78 r9:bf7cfc98 r8:bf7cfc14 r7:00000000 r6:8079cc00 r5:00000003 r4:bf7cfc00 [<8008c220>] (hrtimer_interrupt) from [<80018728>] (twd_handler+0x3c/0x50) r10:807eea3c r9:f4000100 r8:bf022d80 r7:00000010 r6:bf7d5700 r5:bf08a400 r4:00000001 [<800186ec>] (twd_handler) from [<8007c1e8>] (handle_percpu_devid_irq+0x88/0x1a0) r5:bf08a400 r4:8079e38c [<8007c160>] (handle_percpu_devid_irq) from [<8007759c>] (generic_handle_irq+0x30/0x44) r10:807a4cc0 r9:f4000100 r8:bf034000 r7:00000001 r6:00000000 r5:00000010 r4:8079e38c r3:8007c160 [<8007756c>] (generic_handle_irq) from [<800778a0>] (__handle_domain_irq+0x60/0xc8) [<80077840>] (__handle_domain_irq) from [<800094c0>] (gic_handle_irq+0x60/0xc4) r9:f4000100 r8:807eea38 r7:beb5bde8 r6:000003ff r5:000003eb r4:f400010c [<80009460>] (gic_handle_irq) from [<80593534>] (__irq_svc+0x54/0x70) Exception stack(0xbeb5bde8 to 0xbeb5be30) bde0: bf1a1e44 00000000 0000b2cc 0000a7b4 00000000 00000000 be00: ffffffff 01bc9294 bf1a1e40 bf1a1e40 00000001 beb5be44 80592d34 beb5be38 be20: 8009ef44 80592d60 80010013 ffffffff r10:00000001 r9:bf1a1e40 r8:bf1a1e40 r7:beb5be1c r6:ffffffff r5:80010013 r4:80592d60 [<80592d20>] (_raw_spin_lock) from [<8009ef44>] (futex_wake+0xa4/0x194) [<8009eea0>] (futex_wake) from [<800a0ec4>] (do_futex+0x128/0xd80) r10:000000f0 r9:01bc9294 r8:00000001 r7:01bc9294 r6:00000081 r5:00000000 r4:00000000 [<800a0d9c>] (do_futex) from [<800a1ba4>] (SyS_futex+0x88/0x188) r10:000000f0 r9:beb5a000 r8:80010844 r7:01bc9294 r6:00000081 r5:00000000 r4:00000001 [<800a1b1c>] (SyS_futex) from [<80010660>] (ret_fast_syscall+0x0/0x48) r10:000000f0 r9:beb5a000 r8:80010844 r7:000000f0 r6:7ee2e842 r5:01bc92b0 r4:00000000

There doesn't appear to be any pattern, when it occurs and I cannot reproduce it. Any ideas on how to debug this issue?

Thanks,
Ivan

Re: RCU stall on kernel 4.4.240

ivang — Wed, 15 Jun 2022 16:02:48 GMT

HI,

but there is some similar bug? if yes can you share some details for identify the issue root cause?

Regards,

Ivan

i.MX ProcessorsのトピックRe: RCU stall on kernel 4.4.240

RCU stall on kernel 4.4.240

Re: RCU stall on kernel 4.4.240

Re: RCU stall on kernel 4.4.240