Fedora 34: Wifi card crashing randomly

Jacob_Dillon · October 13, 2021, 1:48am

Hi all,
Yesterday, I installed Fedora 34 on my newly arrived Framework DIY edition, complete with AX210 wifi card. Today, I’ve been having issues where the wifi will randomly cut out, and not work again until a reboot. Checked dmesg, and found this:

[ 3889.494065] iwlwifi 0000:aa:00.0: regular scan timed out
[ 3889.495908] iwlwifi 0000:aa:00.0: Microcode SW error detected. Restarting 0x0.
[ 3889.496403] iwlwifi 0000:aa:00.0: Start IWL Error Log Dump:
[ 3889.496407] iwlwifi 0000:aa:00.0: Transport status: 0x0000004A, valid: 6
[ 3889.496412] iwlwifi 0000:aa:00.0: Loaded firmware version: 63.c04f3485.0 ty-a0-gf-a0-63.ucode
[ 3889.496417] iwlwifi 0000:aa:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN       
[ 3889.496422] iwlwifi 0000:aa:00.0: 0x000002F0 | trm_hw_status0
[ 3889.496425] iwlwifi 0000:aa:00.0: 0x00000000 | trm_hw_status1
[ 3889.496428] iwlwifi 0000:aa:00.0: 0x004DA004 | branchlink2
[ 3889.496431] iwlwifi 0000:aa:00.0: 0x004D07EA | interruptlink1
[ 3889.496434] iwlwifi 0000:aa:00.0: 0x004D07EA | interruptlink2
[ 3889.496436] iwlwifi 0000:aa:00.0: 0x000141D2 | data1
[ 3889.496439] iwlwifi 0000:aa:00.0: 0x01000000 | data2
[ 3889.496442] iwlwifi 0000:aa:00.0: 0x00000000 | data3
[ 3889.496444] iwlwifi 0000:aa:00.0: 0x9C4105F9 | beacon time
[ 3889.496446] iwlwifi 0000:aa:00.0: 0xC744BC4E | tsf low
[ 3889.496448] iwlwifi 0000:aa:00.0: 0x0000009B | tsf hi
[ 3889.496450] iwlwifi 0000:aa:00.0: 0x00000000 | time gp1
[ 3889.496452] iwlwifi 0000:aa:00.0: 0xE654A253 | time gp2
[ 3889.496453] iwlwifi 0000:aa:00.0: 0x00000001 | uCode revision type
[ 3889.496455] iwlwifi 0000:aa:00.0: 0x0000003F | uCode version major
[ 3889.496457] iwlwifi 0000:aa:00.0: 0xC04F3485 | uCode version minor
[ 3889.496459] iwlwifi 0000:aa:00.0: 0x00000420 | hw version
[ 3889.496460] iwlwifi 0000:aa:00.0: 0x00C89002 | board version
[ 3889.496462] iwlwifi 0000:aa:00.0: 0x80C4FC04 | hcmd
[ 3889.496464] iwlwifi 0000:aa:00.0: 0x00020000 | isr0
[ 3889.496465] iwlwifi 0000:aa:00.0: 0x00000000 | isr1
[ 3889.496467] iwlwifi 0000:aa:00.0: 0x48F00002 | isr2
[ 3889.496468] iwlwifi 0000:aa:00.0: 0x00C37FCC | isr3
[ 3889.496470] iwlwifi 0000:aa:00.0: 0x00000000 | isr4
[ 3889.496471] iwlwifi 0000:aa:00.0: 0x05CE001C | last cmd Id
[ 3889.496473] iwlwifi 0000:aa:00.0: 0x000141D2 | wait_event
[ 3889.496475] iwlwifi 0000:aa:00.0: 0x00000000 | l2p_control
[ 3889.496476] iwlwifi 0000:aa:00.0: 0x00000020 | l2p_duration
[ 3889.496478] iwlwifi 0000:aa:00.0: 0x00000000 | l2p_mhvalid
[ 3889.496479] iwlwifi 0000:aa:00.0: 0x00000000 | l2p_addr_match
[ 3889.496481] iwlwifi 0000:aa:00.0: 0x00000009 | lmpm_pmg_sel
[ 3889.496483] iwlwifi 0000:aa:00.0: 0x00000000 | timestamp
[ 3889.496484] iwlwifi 0000:aa:00.0: 0x0000503C | flow_handler
[ 3889.496704] iwlwifi 0000:aa:00.0: Start IWL Error Log Dump:
[ 3889.496706] iwlwifi 0000:aa:00.0: Transport status: 0x0000004A, valid: 7
[ 3889.496708] iwlwifi 0000:aa:00.0: 0x20000066 | NMI_INTERRUPT_HOST
[ 3889.496710] iwlwifi 0000:aa:00.0: 0x00000000 | umac branchlink1
[ 3889.496712] iwlwifi 0000:aa:00.0: 0x8045CF40 | umac branchlink2
[ 3889.496714] iwlwifi 0000:aa:00.0: 0x8047E3DE | umac interruptlink1
[ 3889.496715] iwlwifi 0000:aa:00.0: 0x8047E3DE | umac interruptlink2
[ 3889.496717] iwlwifi 0000:aa:00.0: 0x01000000 | umac data1
[ 3889.496718] iwlwifi 0000:aa:00.0: 0x8047E3DE | umac data2
[ 3889.496720] iwlwifi 0000:aa:00.0: 0x00000000 | umac data3
[ 3889.496721] iwlwifi 0000:aa:00.0: 0x0000003F | umac major
[ 3889.496723] iwlwifi 0000:aa:00.0: 0xC04F3485 | umac minor
[ 3889.496725] iwlwifi 0000:aa:00.0: 0xE654A74E | frame pointer
[ 3889.496726] iwlwifi 0000:aa:00.0: 0xC0886264 | stack pointer
[ 3889.496728] iwlwifi 0000:aa:00.0: 0x00ED019C | last host cmd
[ 3889.496729] iwlwifi 0000:aa:00.0: 0x00000400 | isr status reg
[ 3889.496920] iwlwifi 0000:aa:00.0: IML/ROM dump:
[ 3889.496922] iwlwifi 0000:aa:00.0: 0x00000B03 | IML/ROM error/state
[ 3889.496981] iwlwifi 0000:aa:00.0: 0x00008702 | IML/ROM data1
[ 3889.496994] iwlwifi 0000:aa:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[ 3889.497001] iwlwifi 0000:aa:00.0: Fseq Registers:
[ 3889.497063] iwlwifi 0000:aa:00.0: 0x60000000 | FSEQ_ERROR_CODE
[ 3889.497203] iwlwifi 0000:aa:00.0: 0x00440003 | FSEQ_TOP_INIT_VERSION
[ 3889.497328] iwlwifi 0000:aa:00.0: 0x00080009 | FSEQ_CNVIO_INIT_VERSION
[ 3889.497454] iwlwifi 0000:aa:00.0: 0x0000A652 | FSEQ_OTP_VERSION
[ 3889.497580] iwlwifi 0000:aa:00.0: 0x00000002 | FSEQ_TOP_CONTENT_VERSION
[ 3889.497707] iwlwifi 0000:aa:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[ 3889.497832] iwlwifi 0000:aa:00.0: 0x00400410 | FSEQ_CNVI_ID
[ 3889.497970] iwlwifi 0000:aa:00.0: 0x00400410 | FSEQ_CNVR_ID
[ 3889.498100] iwlwifi 0000:aa:00.0: 0x00400410 | CNVI_AUX_MISC_CHIP
[ 3889.498241] iwlwifi 0000:aa:00.0: 0x00400410 | CNVR_AUX_MISC_CHIP
[ 3889.498368] iwlwifi 0000:aa:00.0: 0x00009061 | CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[ 3889.498497] iwlwifi 0000:aa:00.0: 0x00000061 | CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[ 3889.498826] iwlwifi 0000:aa:00.0: WRT: Collecting data: ini trigger 4 fired (delay=0ms).
[ 3889.498832] ieee80211 phy0: Hardware restart was requested

Not sure what further troubleshooting steps to take here. It’s a pretty annoying issue, and has happened twice in the last 90 minutes… any advice? Packages/kernel are all fully up to date (Linux 5.14.10), so I don’t think it’s similar to the unreliability issues that people have been having with some versions of 5.13…

Carlos_Fernandez_San · October 13, 2021, 2:46am

Try kernel 5.12. For me 5.14 didn’t work.

Linux framework 5.12.0-051200-generic #202104252130 SMP Sun Apr 25 21:33:13 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Important stuff

1. Update the firmware from intel website
1. If Wifi 6 doesn’t work well (for me it doesn’t, but it could be my router or who knows what), try 5G.

Jacob_Dillon · October 18, 2021, 2:06am

Thanks for the reply @Carlos_Fernandez_San, fortunately this issue went away upon upgrading to the Fedora 35 beta!

Carlos_Fernandez_San · October 18, 2021, 9:03pm

What exact linux kernel are you using?
Also, have you tried to run a speedtest (the one from google is fine)? For me that firmware issue appears under stress.

Jacob_Dillon · October 18, 2021, 11:46pm

5.14.10-300.fc35.x86_64

Yup! Works completely fine now, even under a speedtest.

Carlos_Fernandez_San · October 19, 2021, 12:35am

The one from Ubuntu is 5.14.0. I saw they have packages for 5.14.13, but they depend on libc6 2.34 (currently 2.33 on Ubuntu 21.04), which breaks locales…

It seems like a mid sized rabbit hole

Andrew_Marshall · October 29, 2021, 12:59pm

See also my post here for how I reliably reproduce what is likely the same issue.

Andrew_King · November 5, 2021, 7:45pm

I am seeing this exact issue on fedora 35 kernel 5.14.15 - disconnecting and reconnecting to wifi seems to fix the issue for a bit until the next crash.

Strangely I don’t think was happening on the beta release of Fedora 35, I can’t figure out what updated to break it though. Downgrading kernels and firmware does not seem to prevent the crashes.

Andrew_King · November 7, 2021, 12:39am

Possible update/observation - I updated my router firmware (I’m using an ASUS RT-AX86U) and have not had the wifi card crash on me since. I am not sure if this is just coincidental or if some wifi card firmware and router firmware interaction was causing the issue (is that possible?). Either way I haven’t seen the issue appear today and I was seeing crashes at least hourly before. I’ll update if anything changes.

EDIT: Nope it crashed again - though it has only been one crash in 24 hours so somthing may have changed with this

Mike_Kelly · November 8, 2021, 4:16pm

I decided to order a Wireless AC card. This issue has more/less rendered my laptop unusable. I literally have to keep a terminal up ready to run a bounce_wifi.sh script every 10 mins or so.

I’m going to monitor @Andrew_Marshall 's bug report here but in the meantime I need stable wifi.

This is what I ordered to replace the Wifi6 card. It comes in later this week, if I remember I’ll post my results for anybody looking to replace their wireless card.

Carlos_Fernandez_San · November 8, 2021, 5:42pm

Andrew’s problem seems more severe than mine. For me having separate SSIDs on my router for each frequency and then connecting only to the 5G only (never use Wifi 6) “solves” the problem (if we call solving to get 5G speeds - to me it’s acceptable, this is actually my first Wifi 6 laptop so I’m not used to anything better yet).

Comfy · November 8, 2021, 7:07pm

I also ordered an 802.11ac card to temporarily work around the issue until the driver/firmware issues are fixed for the AX210 upstream.

Since the Framework supports CNVi, I went with an Intel 9560 radio module, and it works perfectly with kernel 5.14.x on Fedora 35.

Andrew_Marshall · November 8, 2021, 8:36pm

@Comfy Could you confirm if you can reproduce the issue with the AX210 using my reproduction steps from the bug report, but that you cannot reproduce with the same steps using a 9560 module?

Comfy · November 9, 2021, 5:49pm

I can try to reproduce using your steps when I get some time to swap the AX210 card back in. The main issues I was experiencing with it were poor throughput and high latency, and I saw some “hardware reset” panics in iwlwifi, particularly on suspend/resume. I see none of that with the 9560 module though.

Mike_Kelly · November 10, 2021, 6:34am

Reporting back. Before swapping in this wireless card in Fedora with kernel 5.14 I was losing my connection every 10-15 mins. This wireless AC card stayed connected without issues all day and my speed tests increased by 20-50mbps.