Clock stuck at .39Ghz

@reverse_thrust @computerhac
what software are you using to generate those graphs?

I think you are using excel with the output from HWINFO. If so could you send me the log file?

I have been trying to reproduce this in the background for a while and having trouble getting any reproduction to happen with several units I have on hand.

I have a theory for what might be happening, but I would like to confirm it can reproduce on hardware.

Just to reiterate. There are potentially a few causes of this which can be caused by different things.

  1. Clock stuck at 200MHZ. This is usually caused by prochot stuck low. We have found this is usually caused by the USB port EMI sticker issue on early batches.
  2. Clock stuck at 400MHZ for a short time before climbing without prochot- we are still investigating this, but you can try adjusting the battery slider to max performance to see if this solves the issue. Unknown root cause.
  3. Clock stuck at 400MHz with what looks like prochot toggling during specific portions of the battery charge curve. This one we are still trying to reproduce. Unknown root cause.

For item 3, if prochot is being triggered, this will force the CPU to min speed state of 200MHz. however perhaps if it is toggling I suspect the logging software may average this out to something higher.

It would be great to get the complete logs during this event from something like HWINFO that shows:
CPU frequency, Charge %, PL1 Power Limit, PL2 Power Limit, prochot status.

Also what would be useful as well is simultaneously logging from the intel power gadget

When in the failure condition:
If we see the PL Power limits are dropping to a very low value causing the low CPU speed, then this is most likely caused by the DPTF driver incorrectly throttling the CPU.

If we see the the PL power limits remain high, eg 15-28W for PL1, but cpu clocks at 400MHz then we can look further at the hardware prochot sources to try and determine what is triggering the prochot. One theory I have is that the charger IC prochot limit setting in firmware is too close to the input current limit, and some variation in the sense resistors may cause false triggering of the prochot signal at a rapid rate while the control loop is in hysteresis. This may cause some units to exhibit this issue depending on part tolerances, but other units may be ok. If this is the case we can issue a firmware update to fix this for impacted units.

14 Likes

I sent a message with the relevant logs, briefly, as far as I can tell PL power limits remain high in my case. I’ll install Power Gadget and get a log on my next charge cycle. I think my HWiNFO polling frequency is a bit slow by default, but PROCHOT does generally appear to be toggling, transitioning from YES at the start of the core frequency drop, and toggling with increasing frequency to NO as core behavior recovers. As @computerhac noted this behavior is timed right about when the charge rate drops off.

After looking at my data it appears the prochot is triggering when this starts and continues to trigger all the way to the end. Could explain why the clocks aren’t as high on average even at 98% battery. I’ll install power gadget and get some fresh logs tomorrow.

@Kieran_Levin , there is obviously a ton to read through in this thread.

Many of us are experiencing #3, see:

I did some tests after catching up on this thread. I love those graphs by the way. It looks like using a lower powered 30W adaptor does cause the issue to go away for me. It still manages to charge at idle. First, I got a baseline with the framework 60W charger and started charging at around 80% battery, watched as the clock speed dropped to 0.39Ghz. PL1 and PL2 stay at 28W and 60W respectively. Then, I swapped it to the 30W Anker PowerPort II PD using the same USB-C cable and had no issue at that charge state. The Anker model A2321 is rated at 5V 3A, 9V 3A, 15V 2A, 20V 1.5A. The framework unit shows 5V 3A, 9V 3A, 15V 3A, 20V 3A.
Then, I did a screen recording to watch it charge from 30% to 100% and didn’t see any issues.

Maybe this is just Framework telling us we shouldn’t charge over 80% to reduce battery ewaste :slight_smile:

2 Likes

“composite” is two temp sensors on the NVME, one for the flash and one for the controller, combined into a single number for the whole drive.

I don’t know specific form the combination takes, it might be an average or might be whichever-is-highest, or possibly even something else.

2 Likes

I am seeing this behavior (capped at 0.39 GHz) on my batch 2 laptop, running Windows 10 21H1. I think I saw it happening back when I first got it and installed Windows too…

When the Framework charger is plugged in (power profile set to best performance), CPU caps out at 0.39 GHz. Unplug the charger, and the frequency goes back up.

Not sure if this is helpful or relevant, but my laptop had been sitting powered off for around 2 weeks, and it has been getting cold (ambient temp in this room is about 64 F). When I tried to turn my laptop on today, at first it just didn’t turn on at all. (No reaction to the power button, no side LEDs.) Once I plugged in the charger it turned on like normal (other than being really slow), except the clock was not updated (it showed 10pm on 10/12, which is right about when I had powered it down). Had to manually update the time to get NTP to work. Battery charge reported as 60%.

I will check the EMI stickers.

Update: To my eye, the only EMI sticker that isn’t at basically right angles is the top left one. I tried swapping it from my micro SD bay to USB-C: same 0.39GHz capping. Battery is currently reporting at 77%.

2 Likes

All of them are supposed to be a gentle curve, not sharp tight angles pressed tight to the connector. Unless you have the “fixed version” that has an insulator on the underside of the sticker.

If time was off, check the CMOS battery a few folks have had it be loose.

All of them are supposed to be a gentle curve

Understood; what I meant in my comment is that the only one with a gentle curve is the top left. The other three seem to be press-fit against the connectors. But charging through the connector where the sticker looks okay still results in throttling, so unless it’s a shorting issue that affects neighboring ports, it must be something else on the motherboard, or the expansion card, or the charger, or the OS, or BIOS…

1 Like

Remember that each of 2 of the ports are on 1 controller. So the good one shares a controller with a “not correct” one.

I’d make sure that all the stickers are on properly just in case. Basically to nibble away at the possibilities.

First thing I did was get the stickers all correct on my machine (batch 2 as well) as it gave me a “you could get better bandwidth on another port” when I first started to put in modules. Msg went away before I had even started to read the whole thing…

Just hopping in to say that my Batch 4 laptop is also having this issue

EDIT: However, it is very intermittent. Sometimes my frequency gets stuck at 400 mhz sometimes it seems to fix itself. I am on Linux

1 Like

I have also been having this issue with my new Batch 4 laptop (DIY running Pop! OS).

I was not able to charge using the rear left port, but using USB accessories worked fine in the same location. The laptop was generally working fine so I disregarded that as a fluke.

I noticed that if I powered it on without being plugged in, the CPU would be stuck at 200Mhz. Plugging in power would fix it. Also, removing the expansion card in the problem port would also cause the CPU to get stuck to 200Mhz. In that case, removing power would also fix it.

Tonight I followed the instructions to remove and re-apply the EMI sticker on the problem port. This seems to have resolved the issue for me. No more 200Mhz and I can also charge from that port now.

2 Likes

A quick update. After rereading this thread, I decided to follow the steps done by everyone else

Readjusting EMI Stickers
This had no effect on my Batch 4 laptop

Trying different module locations
No effect

Gathering data on the battery
My battery is also at 95% health which is inline to a majority of people in this post. Like what some other people have said, charging the laptop from 0-80ish % had no such throttling issues. However, from 84%-90%, I started lagging. For some reason, the issue disappeared once I charged the laptop past the 90% threshold. I will need to mess around with this a bit more to see if it is consistent.

Some other interesting points
As I was turning on the laptop from a suspend state, I noticed that s-tui was displaying the CPU running at a normal clock speed before getting stuck at 400mhz again after a few seconds.

Also, side note, I am not using Framework’s charger. Instead, I’m using RAVPower’s GaN 61w charger. Works perfectly fine on my previous T480.

EDIT
Now at 90+%, I’m experiencing an interesting issue. Unplugged, my cpu can turbo up to its rated speed. However, on AC, I can only reach 1.2 GHz

I can now also confirm that I also experience a different reaction on clock speed when the AC adapter is online but not charging vs charging. No issues when online but not charging.

UPDATE
Manually disabling BD PROCHOT allows the system to return to base clock. Total package power reported by turbostat is only 15 watts however. In this case, it seems like the 28w sustained turbo is not occuring

Sadly I don’t have a Framework laptop yet. But is it possible to test if this issue happen when letting the battery disconnected? Can the Framework laptop run without battery?

@Timothy_Aponte a firmware update is in the works to enable battery free use of the mainboard

1 Like

What is Manually disabling BD PROCHOT
read further and will wait for Framework team to figure this out.

Using either Throttlestop on Windows or using some hackery in Linux, you can disable BD PROCHOT.

What BD PROCHOT specifically is (from that github link above):

“BD PROCHOT" stands for bi-directional prochot. PROCHOT stands for processor hot. This signal is what initiates thermal throttling so the CPU can slow down and keep from over heating.
BD PROCHOT can cause CPU run at a very low frequency, on ThinkPad E440, which is only 800MHz(max 2.3GHz). The cause of the signal activation usually are aging power adapter, aging battery, or bad sensor on motherboard. Each of cause can activate BD PROCHOT.

What I found with additional testing is that, while disabling BD PROCHOT does allow the CPU to clock up in speed, the whole package is limited to 15 watts as opposed to its typical 28 watt power draw. This causes the Intel Xe iGPU to cripple itself in performance as the CPU uses the rest of the power budget.

So what do you think the fix will be? Some have surmised that the power supply is the problem and a higher wattage supply worked for them. I am in batch 5 and wonder if I should cancel the power supply and purchase an aftermarket one.
thanks
dan

1 Like

I’ve yet to narrow down a hardware issue or a firmware issue.

I’m curious if BD PROCHOT is triggering due to some kind of overheating charging component. I do notice that some components are getting warm when charging but nothing alarming. Maybe it could also be a temperature probe issue?

If it is a charger power issue, I’m not sure why the issue only appears from charging from 80% and up. Below 80% and the laptop works fine on the charger.

This makes me think it could be a firmware issue too.