Thanks. I had already tried that when trying to set up kdump, but it never worked.
I just tried it, and it did reboot after 20 seconds like I instructed watchdogs to do, which is good, but /var/lib/systemd/pstore is still empty sadly…
@Yam
If /var/lib/systemd/pstore is still empty after a sysrq-trigger, it means that the pstore is not configured correctly.
I don’t use arch, so don’t know exactly how to set it up there.
So, it might be worth you doing a bit of investigation on how to set it up in arch and get it working.
For some pointers, with “sudo lsmod” do you have an “efi_pstore” module listed? Do you have the systemd pstore module?
FYI: pstore is automatically setup correctly in ubuntu. I am currently using ubuntu.
Thanks for the info, sadly there isn’t an official wiki page on how to set up pstore for Arch Linux, but from what I gathered it seems to be loaded fine.
I have CONFIG_PSTORE=y (which means it’s automatically loaded and not a module ?), my kernel parameters include pstore.backend=efi efi_pstore.pstore_disable=0 and I have confirmed that pstore is using the efi backend. Yet absolutely no logs seems to be available anywhere.
I had a few more freezes today, and my system automatically rebooted each time except once, where I had to manually power it down. Watchdog seems to be working like 90% of the time, which is weird.
@Yam
It seems that you are half way there.
pstore works like this:
- Kernel crashes but has nowhere to store the crash dump.
- pstore provides that last gasp place to store the crash dump, in a EFI variable.
pstore can fill up your EFI variable space very quickly so a third step is needed - systemd looks in the EFI variable on every boot and if a crash dump is there (from a previous session), systemd copies it to
/var/lib/systemd/pstore and then deletes the EFI variable so that the EFI variable store does not fill up.
I think you might still need the (3) that is the systemd-pstore bit.
Maybe this will help:
https://man.archlinux.org/man/systemd-pstore.8.en
Thanks. I really tried everything, but there’s still no log. I tried to manually load the modules, but it seems that they are already built-in :
systemd-modules-load[2895]: Module 'pstore' is built in
systemd-modules-load[2895]: Module 'efi_pstore' is built in
I enabled the systemd-pstore service. It’s now enabled but didn’t start because Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore)., which makes sense if it has no logs to work with.
Now I triggered a kernel panic manually and the PC rebooted on its own after 20 seconds, which is what I configured the watchdog to do. But still, nothing to be found.
I just had another freeze, and I waited a good 2 minutes before deciding to power off the laptop. It seems that watchdogs simply can’t reboot the laptop under certain circumstances. And still, no logs from pstore.
So could it be possible that the crash is too big to store in the efi variable ? Or that the efi storage is already full for some reason ? I cannot see any other reason why this would not work at this point.
Would enabling sysrq allow you to capture what’s going on? Keyboard shortcuts - ArchWiki
I have some more info that might help track this down. The quote is from someone at AMD:
Quote starts----
Another thing that is really useful is that there is a register in FCH
called S5_RESET_STATUS. If you can get the value from it when this
fails it can point you at where the issue is. I don’t think this patch
landed, but you can see if it works for your system to print the info.
https://marc.info/?l=linux-i2c&m=168089982408414
There should be some public documentation on interpreting
S5_RESET_STATUS somewhere, but it’s slipping my mind where it is.
—Quote ends.
So, If anyone wishes to compile their own kernels and help with this problem, its very worth while applying the above patch.
Normal values:
Cold Power off / Power on:
S5_RESET_STATUS = 0x00200800
Warm reboot:
S5_RESET_STATUS = 0x00080800
Keeping your finger of the power button for 10 seconds to force the power off.
S5_RESET_STATUS = 0x00000800
Suspend:
Nothing output.
So, we are looking for values that are different than those three values.
@lbkNhubert I had sysrq enabled, but it never worked, so I disabled it.
@James3 I’m not sure if I want to apply a patch that I do not even know how to use. I would rather try to get pstore to work at least once to give me a clue.
Are you saying that it didn’t work when you had a crash and needed it, or that it never operated properly, for example, by triggering it when the system was running fine?
I ask because I have it enabled on my system running arch and have been able to use it when needed. Thankfully that has not been the case of late. Fingers crossed.
Both, actually. I never got it to work when the system was fine, and it obviously never worked when I needed it on any freeze. Did you do anything special to make it work other than just allowing the keys ?
I just have it set in my kernel parameters. Then when I needed to use it I always had to fumble around to get the right keys hit to trigger it, then to do the reisub dance. But eventually I would get it and it would work.
That’s what I had as well, I had it setup to 1 I think, but I simply couldn’t trigger it no matter what keys I used (if I remember well the sysrq key is F11 on the Framework 16, or at least it’s supposed to be).
@James3 Damn thanks a lot. That would explain why the keys weren’t working. I admit I did not search any further and simply gave up on it, but I’ll try again.
On a side note, I just had another freeze today while using an external display and dragging a window, and I managed to get a lot in journalctl somehow. The computer did not restart, so I’m thinking that this isn’t a kernel panic, but simply a case where the whole UI becomes unresponsive.
Here’s the log :
kwin_wayland_drm: Pageflip timed out! This is a kernel bug
493277 – Window drag causes "Pageflip timed out! This is a kernel bug" This seems tightly related to what I’m getting, but looking online, there doesn’t seem to be any solution to this. I already got this bug before, but it did not freeze the UI, so it seems fairly random.
From what I can gather so far, I’m almost positive I’m experiencing 2 different kind of freezes : one where the UI freezes, but the kernel doesn’t panic, it’s the pageflip one I just got, and another where the kernel does panic and the laptop restarts automatically after 20 seconds.
I just tested it. On my FW16, I needed to hold down ctrl super (windows key) and alt, then tap print screen (f11), then type the r e i s u b keys. Apologies for the parentheticals, trying to be explicitly clear rather than leave anything open to interpretation. Best of luck, I hope that you are able to get things resolved soon.
I have the same symptom. Sudden freeze with no logs. It’s a hard a freeze, can not even get to tty, and audio seems to be stuck in a loop. If audio is affected is it a kernel freeze or a amdgpu bug ?
It could honestly be both, no way to be certain. I wouldn’t personally know because I rarely have audio activated on this laptop.
@James3 @lbkNhubert Want to confirm I got the sysrq key working by enabling it again using systemd and then pressing alt + fn + F11 + [desired key]. All keys need to be pressed while doing it, but it does indeed work.
I’ll use these keys next time I get a freeze, hopefully I’ll be able to regain control or at least log some stuff.
1 Like
Just had another freeze today, but the PC didn’t restart. I tried every sysrq key possible but absolutely no effect. I can confirm that they do normally work, so could it be that this is a hardware related issue ? Possibly AMD stuff as stated in FW16 Freeze then Reboot (FTR) · Issue #41 · FrameworkComputer/SoftwareFirmwareIssueTracker · GitHub
Use the LTS kernel, that’s all.