RCU Stall detected when connecting USB to server

[pedro_valladare]

We are seeing an issue when connecting our device to a server running VMWare ESXi 6.7.0 VMKernel. If the passthrough is not enabled for the USB port, our device generates the following RCU and I cannot longer type anything at the console. If passthrough is enabled, then it works fine. If I remove the g_serial.ko module , the issue does not occur.  

Here is the output of uname -a for our system:

Linux (none) 4.1.8-rt8+gbd51baf #1 SMP Thu Jun 6 17:15:51 EDT 2019 ppc64 GNU/Linux

and the output of modinfo g_serial:

filename: /lib/modules/4.1.8-rt8+gbd51baf/kernel/drivers/usb/gadget/legacy/g_serial.ko
license: GPL
author: David Brownell
author: Al Borchers
description: Gadget Serial v2.4
depends: libcomposite
intree: Y
vermagic: 4.1.8-rt8+gbd51baf SMP mod_unload modversions
parm: idVendor:USB Vendor ID (ushort)
parm: idProduct:USB Product ID (ushort)
parm: bcdDevice:USB Device version (BCD) (ushort)
parm: iSerialNumber:SerialNumber string (charp)
parm: iManufacturer:USB Manufacturer string (charp)
parm: iProduct:USB Product string (charp)
parm: use_acm:Use CDC ACM, default=yes (bool)
parm: use_obex:Use CDC OBEX, default=no (bool)
parm: n_ports:number of ports to create, default=1 (uint)

modinfo libcomposite
filename: /lib/modules/4.1.8-rt8+gbd51baf/kernel/drivers/usb/gadget/libcomposite.ko
author: David Brownell
license: GPL
depends: configfs,udc-core
intree: Y
vermagic: 4.1.8-rt8+gbd51baf SMP mod_unload modversions

root@(none):~# modinfo udc-core
filename: /lib/modules/4.1.8-rt8+gbd51baf/kernel/drivers/usb/gadget/udc/udc-core.ko
license: GPL v2
author: Felipe Balbi <balbi@ti.com>
description: UDC Framework
depends:
intree: Y
vermagic: 4.1.8-rt8+gbd51baf SMP mod_unload modversions

root@(none):~# modinfo configfs
filename: /lib/modules/4.1.8-rt8+gbd51baf/kernel/fs/configfs/configfs.ko
description: Simple RAM filesystem for user driven kernel subsystem configuration.
version: 0.0.2
license: GPL
author: Oracle
alias: fs-configfs
srcversion: BAC846119DF7A8FAD803348
depends:
intree: Y
vermagic: 4.1.8-rt8+gbd51baf SMP mod_unload modversions

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

INFO: rcu_sched self-detected stall on CPU
INFO: rcu_sched self-detected stall on CPU
0: (5250 ticks this GP) idle=e15/140000000000001/0 softirq=0/0 fqs=24
(t=5250 jiffies g=3340 c=3339 q=11378)
rcu_sched kthread starved for 5226 jiffies!
Task dump for CPU 0:
run R running task 0 3068 3008 0x00000004
Call Trace:
[c0000000ca1b29d0] [c0000000000858d8] .sched_show_task+0xd8/0x164 (unreliable)
[c0000000ca1b2a50] [c0000000000ba970] .rcu_dump_cpu_stacks+0xd8/0x150
[c0000000ca1b2af0] [c0000000000c0364] .rcu_check_callbacks+0x5e0/0x8f4
[c0000000ca1b2c40] [c0000000000c6cec] .update_process_times+0x54/0x94
[c0000000ca1b2cc0] [c0000000000dc6d4] .tick_sched_handle.isra.17+0x5c/0x7c
[c0000000ca1b2d50] [c0000000000dc758] .tick_sched_timer+0x64/0xcc
[c0000000ca1b2df0] [c0000000000c7ff8] .__run_hrtimer+0xc0/0x2ec
[c0000000ca1b2e90] [c0000000000c85a8] .hrtimer_interrupt+0x13c/0x2f8
[c0000000ca1b2fa0] [c00000000001217c] .__timer_interrupt+0xa0/0x200
[c0000000ca1b3040] [c00000000001261c] .timer_interrupt+0x90/0xc4
[c0000000ca1b30c0] [c00000000001c054] exc_0x900_common+0x104/0x108
--- interrupt: 901 at .smp_call_function_many+0x324/0x3b8
LR = .smp_call_function_many+0x2e0/0x3b8
[c0000000ca1b33b0] [c0000000000e2f48] .smp_call_function_many+0x2c0/0x3b8 (unreliable)
[c0000000ca1b3470] [c00000000002ed48] .__flush_tlb_page+0x140/0x19c
[c0000000ca1b3530] [c00000000002c64c] .ptep_set_access_flags+0xb8/0x160
[c0000000ca1b35d0] [c00000000018dbb0] .do_wp_page+0x1ac/0x758
[c0000000ca1b36d0] [c00000000019078c] .handle_mm_fault+0xab8/0x1198
[c0000000ca1b37f0] [c00000000002b874] .do_page_fault+0x398/0x6cc
[c0000000ca1b38c0] [c00000000001d1d4] storage_fault_common+0x20/0x44
--- interrupt: 301 at .handle_rt_signal64+0x164/0x440
LR = .handle_rt_signal64+0x9c/0x440
[c0000000ca1b3c70] [c00000000000a974] .do_signal+0x17c/0x220
[c0000000ca1b3db0] [c00000000000ab98] .do_notify_resume+0x84/0x94
[c0000000ca1b3e30] [c000000000000c4c] .ret_from_except_lite+0x78/0x7c
Task dump for CPU 4:
run R running task 0 3111 3045 0x00000004
Call Trace:
[c000000062f23900] [c000000001117edc] nr_cpu_ids+0x0/0x4 (unreliable)
Task dump for CPU 6:
swapper/6 R running task 0 0 1 0x00000804
Call Trace:
[c0000000f022f9e0] [c0000000f022fa70] 0xc0000000f022fa70 (unreliable)

4: (1 GPs behind) idle=a65/140000000000001/0 softirq=0/0 fqs=24
(t=5302 jiffies g=3340 c=3339 q=11378)
rcu_sched kthread starved for 5279 jiffies!
Task dump for CPU 0:
run R running task 0 3068 3008 0x00000004
Call Trace:
[c0000000ca1b3900] [c0000000ca1b39a0] 0xc0000000ca1b39a0 (unreliable)
Task dump for CPU 4:
run R running task 0 3111 3045 0x00000004
Call Trace:
[c000000062f22f40] [c0000000000858d8] .sched_show_task+0xd8/0x164 (unreliable)
[c000000062f22fc0] [c0000000000ba970] .rcu_dump_cpu_stacks+0xd8/0x150
[c000000062f23060] [c0000000000c0364] .rcu_check_callbacks+0x5e0/0x8f4
[c000000062f231b0] [c0000000000c6cec] .update_process_times+0x54/0x94
[c000000062f23230] [c0000000000dc6d4] .tick_sched_handle.isra.17+0x5c/0x7c
[c000000062f232c0] [c0000000000dc758] .tick_sched_timer+0x64/0xcc
[c000000062f23360] [c0000000000c7ff8] .__run_hrtimer+0xc0/0x2ec
[c000000062f23400] [c0000000000c85a8] .hrtimer_interrupt+0x13c/0x2f8
[c000000062f23510] [c00000000001217c] .__timer_interrupt+0xa0/0x200
[c000000062f235b0] [c00000000001261c] .timer_interrupt+0x90/0xc4
[c000000062f23630] [c00000000001c054] exc_0x900_common+0x104/0x108
--- interrupt: 901 at .smp_call_function_many+0x324/0x3b8
LR = .smp_call_function_many+0x2e0/0x3b8
[c000000062f23920] [c0000000000e2f48] .smp_call_function_many+0x2c0/0x3b8 (unreliable)
[c000000062f239e0] [c00000000002ed48] .__flush_tlb_page+0x140/0x19c
[c000000062f23aa0] [c00000000002c64c] .ptep_set_access_flags+0xb8/0x160
[c000000062f23b40] [c00000000018dbb0] .do_wp_page+0x1ac/0x758
[c000000062f23c40] [c00000000019078c] .handle_mm_fault+0xab8/0x1198
[c000000062f23d60] [c00000000002b874] .do_page_fault+0x398/0x6cc
[c000000062f23e30] [c00000000001d1d4] storage_fault_common+0x20/0x44
Task dump for CPU 6:

[pedro_valladare]

Hi Igor,

Thank you for your prompt response. However, I just realized that I might have posted this in the wrong group. We are using Qoir T4240 Power PC based system.

Is it possible for you to forward this to the correct group , please?

Thanks and regards,

Pedro

[igorpadykov]

Hi Pedro

seems most simple is to create new thread in

QorIQ Processing Platforms 

or support live chat:

Support | NXP 

Best regards
igor

[igorpadykov]

Hi Pedro

hanging may be due to ddr memory errors so one can try to run ddr test and

rebuild image with new calibration settings found from ddr tester

i.MX6/7 DDR Stress Test Tool V3.00 

Linux  4.1.8 version is not officially supported by nxp, one can try with nxp linux

from codeaurora.org/external/imx/linux-imx repository:

linux-imx - i.MX Linux kernel 

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[pedro_valladare]

Hi Igor,

I am not sure how to move this to the T4240 based processors group. Could you help to move it to the correct group please? We are still experiencing this issue.

Thanks and regards,

Pedro