Fedora35 unbootable after today's updates

Hello FW community!

Today my perfectly working laptop became completely inaccessible. It is on the latest kernel 5.16, Fedora 35 KDE, has fingerprint enabled and working.

I did two things since the last boot: applied updates (only from today, was up to date) and installed but not configured snapper tool for btrfs (to try to make filesystem snapshots). I ran “snapper list”, it complained about the missing config. I did nothing else. I do not think this can hose the whole btrfs. I still can read the files when I boot from live iso and mount.

When I try to boot normally (select kernel), it shows “Framework” and Fedora labels, shows spinner for some time then Fedora disappears and nothing happens. No keyboard reactions, nothing works. No login screen shown, just big Framework on the black bkg. But when I press the power button, I see the spinner and Fedora label again, then it shuts down. So it’s not hung or dead, it reacts to this one button.

When I tried the rescue boot entry, I got “root user locked” error and no way to enter the console. I tried to fix that by booting live cd/usb and doing “passwd root” after mounting the root and boot partitions (found Fedora doc). This command fails with an auth error. I guess it wants the fingerprint but can’t read it. After this manipulation I got the root password prompt in the rescue boot, but no passwords worked. Still cannot get into the rescue console.

I made a clonezilla full disk backup a couple of days ago, so this is a way out. But I can’t access it for several days due to traveling.

I am out of ideas at the moment and would appreciate any help.

Thank you!
Al.

What happens if you edit the regular boot entry and remove quiet splash?

Duh… Forgot to do that.
I had only “quiet”, removed that. Got log scrolling, then again Framework and Fedora (splash?), then about two dozen lines of some error showed up for split second, and now is a blinking cursor on black bkg. I don’t think I can be that fast to even take a picture of the error log, will try video.

Removed “rhgb quiet” from the boot options. Now all the logs are shown, no errors.
Last two lines:
Starting Hold until boot process finishes up…
Starting Terminate Plymouth Boot Screen…

That’s it.

I tried ctrl+alt+f2, got text login prompt. Cannot login as my user or root, no passwords work.

Tried again exact same steps as above, got blinking cursor again. Then it also disappeared, and ctrl+alt+f2 did not work. This is weird…

If you add init=/bin/bash to your kernel command line, that should drop you to a root shell (see kids, this is why you enable full disk encryption!). You can then do a mount -o remount,rw / to get a primitive root shell with read/write access to your root partition. Of course, you could also mount other partitions if necessary to debug.

That being said, I’ve never seen anything like this.

2 Likes

OK, got into that shell. Updated root password, and it says success.
Can I disable fingerprint scanner from this shell too?

Well, not so fast.

I tried 3 times, doing “sync” after “passwd root”, doing “mount -o remount, ro /”.
No matter what, when I do “exit”, I get kernel panic. And password changes do not stick, can’t get into the rescue shell.

Well, I didn’t like that Fedora anyway…
Wanted to try your Debian sid way, @Chiraag_Nataraj… :wink:

I’ll try to reinstall Fedora while keeping home partition first, then just wipe it and do Debian.

Cool :wink: Lemme know if you have questions about Debian sid, or post over on this thread.

Snapper could absolutely be the problem here. It seems that snapper and btrfs have had a slew of issues if not configured properly on Fedora. It also seems like the GRUB that Fedora uses doesn’t play well with it nicely either.

Example:

My best guess is that snapper could have copied over it’s default config and is using that when rebooting, which could be causing a number of problems that could cause your boot to hang.

However, I also had issues with the relatively recent 5.15.5 kernel update they pushed that broke many of my KDE functionality as well. I will be happy to report back if this 5.16 update borks my system as well.

Does this issue persist if you try to boot into an older kernel version, or just the most recent 5.16?

I have 5.14 and 5.15 hanging exactly the same way.

If that was snapper, then it explains why I can’t reset root password as well.

Can I do anything about snapper from a live USB boot?

Potentially. I’m only moderately experienced with Linux myself so I’m sorry if this isn’t the most helpful answer.

If you can mount your Fedora partition you could either try and edit snapper’s config based on some of the things mentioned in the previous reddit post I linked and some other forums. You could also see if you could uninstall the snapper package and see if that resolves it as well. Failing that you could at least rescue any important files from your partition to an external drive and do a clean install.

So I could not rescue this Fedora instance. After home folder backup I tried Debian sid.

It was short and catastrophic, @Chiraag_Nataraj. Install went fine, switch to sid too. Then I used the installer to add KDE. On the next reboot I tried to select KDE on the login screen. It resulted in black screen, no desktop. After hard power down and reboot, same thing.

So, I abandoned Debian and on SUSE Tumbleweed now. And I love it! Everything works, hibernate too. And it has working Snapper, which makes snapshots after every app install or update. Love this feature in particular!

Huh, that’s bizarre. I’m sitting here on sid using sway across multiple devices just fine (completely different hardware between the two machines I’ve setup too).

Glad you found a distro that works for you, though!