A few things to try out, since we know the issue (basically your CPU is hitting 105C and trigger PROCHOT), a few solutions:
Since you’re only hitting this after a few hours of gaming/being fully thermally saturated, have you considered using a laptop stand or an active cooling pad? Honestly, anything to make sure the fans and vents aren’t being blocked might be enough to make sure it doesn’t happen.
Make sure you are running thermald and adjust the thermal-conf.xml and lower the TripPoint Temperature. You should be able to force your CPU to throttle before it gets to the danger zone.
You can also run an app like MangoHud which will show you your CPU and GPU temps (or more detailed tools like s-tui, turbostat, or sensors on a remote terminal) if you need to be more aggressive w/ your thermal mitigation measures.
I suppose I could, but I like to game in various places, often with the laptop on my lap (in bed, sometimes, even), and ensuring I have the thermal pad nearby and ready seems like a pain. (Yes, I know, I’m a little weird.)
Ah, will give it a try! I just installed throttled and was going to see if that helped. But seems like thermald might be easier to configure.
I’ve been using s-tui for the past few days and it’s been really useful to help me see what’s going on.
Another useful tool for gaming would be an fps limiter.
Get GOverlay and limit your CPU to e.g. 30 fps.
Your system will now only render the 30 fps you have set, even if it would be capable of rendering more. This will keep your system cool(er) and even get you more consistent frame times
Hmm, I think I spoke too soon. My attempt to use thermald resulted in triggering the 400MHz issue while the laptop was sitting in a cool room on a cool glass table, with all temperatures in the 40-45C range. I waited over an hour afterward, and eventually had to shut off the laptop and leave it powered down for 10 minutes (after 5 minutes, it was still throttled when I powered up) to get it to go back to normal. Back to giving throttled a try. I think I just don’t know how to configure thermald correctly; seems more complicated than it needs to be (thanks, Intel…).
Edit: nevermind after another half hour or so, one of the core temperatures spiked to 100C, and now all cores are stuck throttled down to 400MHz again.
Ok, finally I think this is solved. After going through a torturous amount of troubleshooting steps with Framework Support, they finally decided to send me a mainboard replacement. I’ve been testing 100% CPU loads for the past hour and a half or so, and I haven’t encountered any throttling.
I’m feeling pretty confident about this because of how drastically different the thermals look with the new board vs. the old. With two simultaneous kernel compiles (using make -j20; make clean over and over in a loop), with the laptop on a glass table (bottom vents unobstructed), temperatures for all cores are stable, and max out at around 71C. If I block the bottom vents (laptop on my blanket-covered lap), temps slowly creep up to about 85C and stay there. Touching the bottom of the laptop at 85C is a little uncomfortable, but doesn’t actually cause pain like with the old board.
With the old mainboard, this kind of load (even with the vents unobstructed) would cause unpredictable temperatures, constantly jumping around to various values between 80C and 100C, without settling to any particular stable temperature.
I am a bit curious as to what’s wrong with the original mainboard. Support had me ensure the fan was clean and working; re-apply thermal paste; remove expansion cards and even RAM, storage, and the WiFi card; and reset the mainboard. The BIOS version on the new and old boards is the same (3.05), and BIOS and OS settings around power management etc. haven’t changed. A visual inspection of the old board doesn’t show any physical damage that I can see.
So, if you’re running into this problem, and you’ve gone through all the troubleshooting steps, I’d suggest you ask your case to be escalated to someone who can authorize a mainboard replacement. Hopefully that helps others as well.
This thread is getting pretty long and goes back a bit, and there are a few things that deviate outside of a vanilla install. If you do not have an active ticket, please create one and link to your posts.
My 12th gen does not exhibit this behavior, but I also keep to a pretty vanilla (guides specific) installation.
Support had me test on the “blessed” Ubuntu LiveCD linked from Framework’s website, and I was able to easily reproduce the issue, both on my original board, and the replacement board that was shipped to me.
(Regardless, if the Framework laptop cannot run a vanilla Debian stable install – I’m now running stable since testing was promoted to stable in June – that’s… pretty atrociously messed up.)
Not sure if my support ticket is still “active”; they had me perform a bunch of troubleshooting steps, and I got shuffled back and forth between several support people who kept asking me to do the same things I’d already done and had provided results for. At this point I’m not sure what else to do on that support ticket, as they’ve given me no further instructions. It’s been incredibly frustrating, to be honest.
Okay, if this is happening on a replacement board and we cannot replicate it. That does put us in a rock and a hard place as something else we cannot duplicate is happening.
We do not test again distros outside of what we have listed. We do however, have multiple pinned community Debian guides.
I would reply to to your last email from the ticket and see what the suggested next steps would be. I imagine if you were sent another board and are still seeing the issue, it would need to be elevated to engineering as we cannot replicate the environment affecting two separate boards (yours and the replacement).
I’m out of town for the next couple weeks, but will do so when I get back home.
Would it not make sense for me to return one of the two mainboards I have that exhibits the problem, and then your support or engineering team can attempt to reproduce it on that board?
Hi everyone,
I encounter the same problem regularly as well. Since I use an eGPU I tend to play a bit heavier games that do not take this throttling lightly: They stutter, freeze and/or crash and sometimes even take down the whole system. I reported this initially in another thread because I didn’t monitor the CPU frequencies back then (htop doesn’t show them): However using s-tui I can monitor that before every crash/freeze the frequency of all cores bumps down to 400 MHz and stays there even after the crash. It can take some time (even through reboots) until the CPU recovers, sometimes it takes half a minute, sometimes more than 10 minutes.
Reproducing this issue is a bit cumbersome, since to me it only occurs after a certain amount of time in the game (0,5-2 hours depending on the game and whether I use the iGPU or not). What I find particularly strange is that in the moment of throttling the CPU isn’t even extremely hot: With my eGPU it sits around 90° up to 95° tops even right before the crash. If I do other CPU-heavy work (which don’t last as long) like compiling code, the CPU can get up to 100° hot without any core throttling down to 400MHz. The only difference I can think of is, that after a 2 hours gaming session the heat has spread over the whole laptop (to other components and the outside of the housing which is quite hot at this point). Maybe this somehow triggers this throttling instead of the temperature inside the CPU? I will definitely do some more testing on this, maybe even on Windows (if this is indeed a Firmware issue, then Windows should be affected as well)
Since we do not test against eGPUs, we cannot speak to this.However, if you see throttling, please do test on Windows well for the eGPU. Linux eGPU handling is a mixed bag, so it can be helpful to have a comparison.
Hi @Matt_Hartley, I again put this off out of frustration, but finally emailed again on the support ticket 3 weeks ago, and again 2 weeks ago, after receiving no response. Any idea why Framework Support seems to be ghosting me?
Your ticket is in with the engineering escalations, I have added a note indicating that you are on the 3.08 BIOS. What games are you trying to play and with which distro/kernel?
My usual game reproducers are Left 4 Dead 2 and Stellaris. Sometimes I can go an hour or so with L4D2 without seeing it occur, but with Stellaris it usually repros within 20 minutes or so. I can repro faster if the vents on the bottom of the laptop are partially blocked by my lap, though it still happens soon enough if it’s on a hard surface like a table.
These are not particularly “strenuous” games; they play just fine on my older 2018 Dell XPS 13.
I’m currently running Debian testing (trixie), kernel 6.6.13. This also occurs on Debian stable (bookworm), any kernel in the 6.2, 6.3, 6.4, 6.5, and 6.6 series, and likely older.
During previous support interactions, I’ve also reproduced this using the Ubuntu 22.04 LTS image linked from Framework’s Linux page, by creating a tmpfs volume and compiling the Linux kernel (with make -j20) over and over.
The OS doesn’t seem to matter once it’s in this state: I can reboot the laptop and enter BIOS settings or the GRUB prompt, and if it’s still throttled I can see sluggish cursor movement and screen redraw.
I’ve disabled turbo boost (both in the BIOS and in sysfs on Linux, and in tlp’s config). I’ve also experimented with throttled, limiting TDP before doing some “softer” throttling, but that hasn’t helped. I’ve also run through support’s suggestions to try removing and shuffling the RAM around, as well as removing the NVMe drive (running the OS off a USB stick) and Wifi card… plus a lot of other stuff they’d asked me to do.
Appreciate the update. We’ll need to see what the escalation team is able to sort from this. In instances where reproduction is tricky, this becomes harder.
On the 16th, we emailed you with thoughts from our engineering team. I’ve added in your latest feedback about the games and tmpfs volume into the ticket as well.
You were asking in another thread what thermal paste I used to solve this issue for me. It’s this noctua thermal paste from amazon. I took my time and was very deliberate about removing the old thermal paste. I even used a little isopropyl alcohol on a q-tip to completely remove it. Then I was pretty generous with applying the new paste.
Other than that, all I did was to use compressed air to blow out the fans. Hope this helps.
I haven’t had the opportunity to try new thermal paste, but I’ve been playing with a possible workaround, at least for gaming: restricting the GPU’s max clock frequency. This seems to be more or less working, though at the expense of worse performance in games, and the need to reduce quality settings in the games themselves.
On Linux I set the values in both /sys/class/drm/card0/gt_max_freq_mhz and /sys/class/drm/card0/gt_boost_freq_mhz to something lower. Looks like they default to 1450. I started by dropping them down to 650, and have been inching them back up, 100MHz at a time, with so far good results up to 950MHz.
My admittedly qualitative test is to stretch out on my couch, put a blanket over myself, the laptop on top of my lap with the blanket under it, and start playing Civilization VI. At the default of 1450MHz, the throttling kicks in after 10 minutes or so. At 950MHz and below I was able to play for several hours with no issues.
I’m still testing (next stop: 1050MHz), so hopefully I still have room to run it faster here. This is a decent workaround: I’m happier to be able to play games at all, vs. having to save and quit every so often when the laptop decided to misbehave (especially a problem with online multiplayer games). But it’s still pretty lame that I have to live with worse perf and lower graphics quality for this to be usable.
And this isn’t a complete workaround. As I’ve mentioned, I can trigger this just by running Linux kernel compiles at full-tilt, even when the GPU is more or less idle.
Update: 1050Mhz was too high. I did manage to play for a about 3 hours before it throttled and got stuck. Will try bumping down to 1000MHz and see if that’s stable. Otherwise it might be 950.
Update2: 1000MHz was no good either; this time it throttled after an hour and a half or so. Back to 950.
Running on an i7-1260p, 64GB RAM, 1TB NVMe, 3.08 BIOS, under Fedora 40.
I changed the thermal paste to a Noctua NT-H2 and cleaned up the fan, the improvement is marginal.
I have disabled Hyper-Threading, yet when is under load (compiling for a couple of minutes) it reaches above 95°C-100°C it starts throttling to 400 MHz and it takes almost half an hour to return to its normal state (when in idle).
The only solution I have found is to limit the CPU frequency using cpupower to 1.8GHz: sudo cpupower --cpu all frequency-set --max 1.8GHz this way the temp is around 80°C under load but at least it does not throttle.
This sucks big time since I select an i7 to have a powerful processor, yet I have to limit it to behave like an i3 (or less).
I don’t know why the cooling system from 13in laptops (in general) is awfully bad, I know about the space constraints, but the way the fan is positioned is just a poor design from every single vendor, airflow is pretty bad and the heatsink too small for a power chip.
Note for my future self: Buy the lower-end CPU available, it will work the same as a high-end CPU.
after another half hour or so, one of the core temperatures spiked to 100C, and now all cores are stuck throttled down to 400MHz again.