Hi Everyone,
I’m working on diagnosing some issue that started overnight causing one of my CPU cores to always be stuck at 90%.
It appears some kworker process is the culprit. Dumping kernel backtraces, I see several USB suspend/resume references in the callstack on the stuck CPU core. Searching online, I have found an old but similar issue. The suggested work around was to disable usbcore.autosuspend via grub. This appears to mitigate the issue for me as well.
Has anyone else experienced this issue?
This work around generally works for me, but comes with increased battery drain since USB cannot be suspended automatically. Ideally I would like to find the true root cause so I can keep USB autospend enabled.
I tried rebooting with an older kernel build, but the issue was still seen. Removing all expansion cards and resetting my BIOS settings also made no difference.
System Details
Framework Laptop i7-1165G7 DIY Edition
BIOS 3.10
Dual Boot: Fedora 36/Windows 11
Kernel Version:
Linux 5.19.9-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 15 09:49:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Kernel Backtrace:
[ 59.343321] NMI backtrace for cpu 1
[ 59.343323] CPU: 1 PID: 94 Comm: kworker/1:1 Not tainted 5.19.9-200.fc36.x86_64 #1
[ 59.343325] Hardware name: Framework Laptop/FRANBMCP0B, BIOS 03.10 07/19/2022
[ 59.343325] Workqueue: pm pm_runtime_work
[ 59.343329] RIP: 0010:_raw_spin_unlock+0x0/0x30
[ 59.343331] Code: 90 5b c3 cc cc cc cc 0f 1f 00 0f 1f 44 00 00 c6 07 00 0f 1f 00 48 8b 3c 24 be 01 02 00 00 e9 b7 7b 36 ff 0f 1f 80 00 00 00 00 <0f> 1f 44 00 00 c6 07 00 0f 1f 00 bf 01 00 00 00 e8 5b 52 39 ff 65
[ 59.343333] RSP: 0018:ffffb236004dfbb0 EFLAGS: 00000002
[ 59.343334] RAX: 0000000000000001 RBX: ffff9fe288bfd1c0 RCX: ffff9fea0fa7a805
[ 59.343335] RDX: ffff9fea0fa71368 RSI: ffff9fea0fa71368 RDI: ffff9fea0fa71340
[ 59.343335] RBP: ffff9fea0fa7a800 R08: ffff9fea0fa71368 R09: ffffffff9ae5cef0
[ 59.343336] R10: 0000000000000000 R11: 0000000000000000 R12: ffff9fe281abf600
[ 59.343337] R13: 0000000000002000 R14: 0000000000000001 R15: ffff9fea0fa71340
[ 59.343338] FS: 0000000000000000(0000) GS:ffff9fea0fa40000(0000) knlGS:0000000000000000
[ 59.343339] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 59.343339] CR2: 00007f93c0008008 CR3: 000000011b252004 CR4: 0000000000770ee0
[ 59.343340] PKRU: 55555554
[ 59.343341] Call Trace:
[ 59.343342] <TASK>
[ 59.343343] __queue_work+0x1da/0x450
[ 59.343347] queue_work_on+0x37/0x40
[ 59.343348] kick_hub_wq.part.0+0x50/0xd0
[ 59.343351] hub_activate+0x627/0x840
[ 59.343353] hub_resume+0x23/0xe0
[ 59.343354] usb_resume_interface.constprop.0.isra.0+0x83/0xc0
[ 59.343358] usb_suspend_both+0xcd/0x230
[ 59.343360] usb_runtime_suspend+0x2a/0x70
[ 59.343362] ? usb_autoresume_device+0x50/0x50
[ 59.343363] __rpm_callback+0x41/0x170
[ 59.343365] ? usb_autoresume_device+0x50/0x50
[ 59.343367] rpm_callback+0x5d/0x70
[ 59.343368] ? usb_autoresume_device+0x50/0x50
[ 59.343369] rpm_suspend+0x10a/0x6f0
[ 59.343371] ? _raw_spin_unlock_irqrestore+0x23/0x40
[ 59.343372] ? __pm_runtime_idle+0x45/0x100
[ 59.343374] ? usb_runtime_resume+0x20/0x20
[ 59.343375] __pm_runtime_suspend+0x38/0x100
[ 59.343377] ? usb_runtime_resume+0x20/0x20
[ 59.343378] usb_runtime_idle+0x31/0x40
[ 59.343380] __rpm_callback+0x41/0x170
[ 59.343381] rpm_idle+0x1fc/0x2f0
[ 59.343383] pm_runtime_work+0x80/0xa0
[ 59.343384] process_one_work+0x1c4/0x380
[ 59.343386] worker_thread+0x4d/0x380
[ 59.343387] ? _raw_spin_lock_irqsave+0x23/0x50
[ 59.343389] ? process_one_work+0x380/0x380
[ 59.343390] kthread+0xe6/0x110
[ 59.343392] ? kthread_complete_and_exit+0x20/0x20
[ 59.343394] ret_from_fork+0x1f/0x30
[ 59.343397] </TASK>
Stuck CPU Core: