Stuck in Emergency Boot Mode After Crash

[Nicholas_Chen]

My laptop crashed while I was browsing YouTube with Firefox, everything just froze, this was right after a system update. After I rebooted, I became stuck in Emergency boot mode, and can’t get out. Here’s what journalctl returns:

I noticed there were efi errors, and of course there is the highlighted issue:

ACPI BIOS Warning (bug): Optional FADT field Pm2ControlBlock has valid Length but zero Address: 0x0000000000000000/0x1 (20230331/tbfadt-615)

From what I can tell, this seems to be a firmware issue, but before I update the firmware I wanted to check if there’s anything else I should do. Any help would be much appreciated.

Specs
System: AMD Ryzen™ 5 7640U
Storage: WD_BLACK™ SN850X NVMe™- M.2 2280 - 500 GB
Memory: DDR5-5600 - 16GB (2 x 8GB)
Distro: Fedora 39 (KDE edition)

[DHowett]

@Nicholas_Chen

After I rebooted, I became stuck in Emergency boot mode, and can’t get out. Here’s what journalctl returns:

Please share a screenshot of the end of the journal. The error here is not causing your boot to fail, and it’s likely that an error at the end will be more enlightening.

(If you’re not familiar with the pager, you can use Shift+G to quickly jump to the end.)

[Nicholas_Chen]

Oops my bad, here’s the end of the journal, thanks for the tip.

[Chad_Nelson]

It looks to me like your BTRFS file system is damaged. Probably not an actual hardware problem, just the file system itself.

Disclaimer: I am NOT an expert on repairing file systems, for all that I’ve been running Linux for years. I keep constant recent backups of my /home hierarchy, so on the rare occasion that something like this happens and I can’t easily recover, I just reinstall the OS and copy my latest data backup back to the system, which immediately gets me 95% of the way back to productive. Haven’t actually had to do that for years, but I’m always prepared for it, from long experience.

With that out of the way: the first thing you should do is copy all of your important data to a backup drive. That probably just means your home directory, if the machine isn’t set up as a server (server programs often store data under /opt or /etc too). If you don’t have an up-to-date backup, create one NOW, before ANYTHING else – copy everything manually if you have to (look at the cp -r command, it will copy the directory you point it to and everything below it). Make sure that you have any important hidden files and directories (whose names start with a period) as well. Double-check that any critical data files have copied correctly before moving on to the next step (md5sum or sha1sum makes that a lot easier), because you have to assume that any repair attempts could wipe out your drive’s contents.

Once you have a safe backup, and have dismounted the backup drive, you’re protected from any mistakes and you can try to fix your main drive. Actually fixing it will probably be either simple or impossible, there isn’t much in between. The btrfs-check program will try to identify the problems, and might be able to fix it (talk to an expert, which again I am not). If so, great – you’re in business again!

If not, the only answer may be to reformat the drive and reinstall the OS and your data. Annoying, but if you have all of your data backed up, that’s all it is – annoying, not disastrous. It usually takes me a couple hours to reinstall everything, as opposed to a couple days when I was running Windows as my main OS.

If, after you reinstall everything, the drive suddenly throws you into this state again, you might have a physical SSD problem that requires replacement. Unlikely, but it’s not impossible.

The only other thing I can do is wish you luck. Feel free to ask if get stuck somewhere.

[Nicholas_Chen]

Thank you so much for the response. It seems pretty daunting but I’ll attempt the back up and drive repair later.
To begin, I’ve never mounted a drive myself before, and I assume I will need to mount the external drive to make the backup. Can I do all the standard things necessary to mount a drive while I’m stuck in emergency mode?

[Chad_Nelson]

It’ll be a learning curve then, but shouldn’t be too steep. Yes, you should be able to do everything you need in “emergency mode,” so far as I know.

[TJ1]

@Nicholas_Chen

Can I do all the standard things necessary to mount a drive while I’m stuck in emergency mode?

Yes.

@Chad_Nelson

The btrfs-check program will try to identify the problems, and might be able to fix it (talk to an expert, which again I am not. If so, great – you’re in business again!

Be extremely careful with any btrfs repair programs. Also, not an expert, but here’s a warning from the man page:

    Warning
           Do not use --repair unless you are advised to do so by a developer or an experienced
           user, and then only after having accepted that no fsck successfully repair all types
           of filesystem corruption. Eg. some other software or hardware bugs can fatally damage
           a volume.

[Nicholas_Chen]

Noted. Thanks for the advice. After I manage to make a back up I might just reinstall the OS like Chad_Nelson says they do, all I really care about right now are my files. However, if I were to run btrfs-check and something goes wrong, I can always just reinstall the OS, right?

[Jonathan_Haas]

@Nicholas_Chen

However, if I were to run btrfs-check and something goes wrong, I can always just reinstall the OS, right?

Sure.

[TJ1]

Yah, re-install is the most likely outcome of this scenario.

Hopefully you can go into single-user mode and mount btrfs read-only even with the whatever has gone wrong and get your data. Everything else can be sorted out later.

[Ulmondil]

@Chad_Nelson

you should be able to do everything you need in “emergency mode,”

You might be able to do everything from “emergency mode” but my advice is to boot a live system from usb. This way is easier, you have internet and GUI, and can install whatever utility you might need.

[Nicholas_Chen]

Hello everyone, right now I’m in a fedora39 live usb and am trying to back up my home directory. However, I’m not quite sure how to do it. I’m trying to locate my home directory, but don’t know where it’s mounted.
My plan was to use this command (where /mnt/external-drive is where I mounted my external SSD)

cp -r /home /mnt/external-drive

But I’m pretty sure that because I’m in a live usb, /home will be the home directory of the live instance, is that correct? So, assuming my laptop’s home directory is in /dev/nvme0n1p3, does this mean I have to mount it as well? Say that’s what I do, how will I point to the home directory within the drive?

[tom_chiverton]

If you connect it, it should show up in the file browser automatically.

[Nicholas_Chen]

Sorry, I ended up fixing that, I edited my post with my current question. Thanks for the help though!

[tom_chiverton]

Hopefully someone who knows the Fedora live CD will be able to say in detail, but again in the file manager you should be able to see your partitions and mount them. You can then just copy and paste your home folder to the external drive.

I’d never touch btrfs with a barge pole myself, I had a similar nightmare last year. Certainly wouldn’t recomend as the default filesystem for anything you care about.

[Nicholas_Chen]

I see, I tried mounting the partition as usual, but got this instead

mount: /home-dir: can’t read superblock on /dev/nvme0n1p3

After checking dmesg, this was the output (it’s very long so I photographed where all the red text started to appear)

Then I checked the drive with fdisk and it returned this

[tom_chiverton]

OK, now that indicates a deeper drive issue, which might explain why the BTRFS filesystem is ill.

You’ll need to either attempt to fix it in place, or dump it off and fix it another way.

There are no good GUIs for this. I was able to rescue my broken BTRFS using The end of btrfs | On:

• /dev/mapper/vgkubuntu-home would be what ever the device for /home-dir is
• /mnt would be the path to your mounted external drive.

If that doesn’t work, you can at least try and dump the raw data for recovery later: dd if=/dev/where-your-btfs-is of=/where/your/usb/is/btrfs.dd

I’d want to do some stress tests on that drive, or check it is correctly seated in the machine, before continuning to use it. Is it new ?

The tool for that is smartctl, though nvme may also be able to shed some light too : SMART tests with smartctl - Thomas-Krenn-Wiki-en

How much do you care about the files ?

[Nicholas_Chen]

Honestly, I’m not too devastated if I lose my files, though I do have one months worth of school notes on it, so it would still suck to lose them.
Another thing to note, I checked the broken drive with smartctl like you suggested and there doesn’t seem to be anything wrong with it, in Media and Data Integrity Errors it returned 0.

I do plan on checking if the drive is seated correctly or not (I have had this new laptop for four months) but before I did anything I wanted to make a back up, which is what I’m having trouble with right now.

[tom_chiverton]

dd is your friend. This copies the raw bytes off the drive aka images the partition. You can then work (on a copy !) of this as a virtual drive to try various recovery methods, mean while reformat the drive from scratch, with something boring and well tested, like Ext4.

Hopefully a leason well learnt about backups, sadly. We’ve all done it

[Nicholas_Chen]

Before I do anything else, I tried to look up ways to fix the superblock issue and found this guide where they install btrfs-progs and then run sudo btrfs rescue super-recover -v /dev/xxx
Is this a good idea to try, or should I dump the drive first?
Also for dumping the drive, the command you provided shows the drive being copied into a .dd file. After I have that file, how would I perform tests/recoveries?