You could check if this also happens on officially supported Linux Distros/Windows.
Also check your system log to see what messages it prints when the TB dock disconnects.
What BIOS are you on? If it is not the 3.06 Beta, then yes I had a lot of strange dock behavior. This continued until I updated to the 3.06 Beta. Since then I have had no issues. Admittedly I am on Fedora which is supported, but weird dock behavior is often BIOS related.
So, in the system journal I see entries like this at the start of the disconnect:
un 26 19:39:05 fw01 kernel: usb 3-2: USB disconnect, device number 4
Jun 26 19:39:05 fw01 kernel: usb 3-2.1: USB disconnect, device number 5
Jun 26 19:39:05 fw01 kernel: usb 3-2.1.1: USB disconnect, device number 9
Jun 26 19:39:05 fw01 kernel: thunderbolt 0-0:1.1: retimer disconnected
and
Jun 27 10:37:36 fw01 kernel: thunderbolt 1-0:3.1: retimer disconnected
Jun 27 10:37:36 fw01 kernel: usb 3-5: USB disconnect, device number 16
Jun 27 10:37:36 fw01 kernel: usb 3-5.1: USB disconnect, device number 18
Jun 27 10:37:36 fw01 kernel: usb 3-5.1.1: USB disconnect, device number 21
Jun 27 10:37:36 fw01 kernel: usb 3-5.1.2: USB disconnect, device number 22
Jun 27 10:37:36 fw01 kernel: usb 3-5.1.2.1: USB disconnect, device number 23
I don’t really see anything that indicates why the disconnect occurs, just that it has happened. dmesg contains the same entries as well.
I’m using the 3.06 Beta BIOS, installed with fwupd.
Have you tried connecting on the other side of the laptop? I have seen this elsewhere with a partial BIOS update. One of the controllers get updated and the other does not, then issues with the retimer follow.
Is your battery charge limit set below 100%? If so, welcome to the club. Kick it back up to 100 to enjoy stability while destroying your battery’s longevity: External display loses signal when charging via docking station
It happens on windows, linux, and FreeBSD so I suspect its a bug in framework’s firmware.
4 Likes
I tried both sides, but get the same issue.
Yes it is, or rather, it was. I’ve bumped it up to 100% and I’ll see what happens over the course of the day. I’ve run two CPU intensive builds so far without a problem, so it’s looking promising.
EDIT: I’ve had a few hours of thunderbolt stability now. So I’m fairly confident that the workaround of setting the charge limit to 100% works.
5 Likes
I really hope this gets addressed in a Framework firmware update.
That’s sad to have to slowly destroy the battery (keeping it plugged for long durations with a 100% limit) in order to gain stability.
The problem is even worse with a Nvidia eGPU, because the nvidia kernel modules do not support hot-unplug, and each disconnection leads to system instability and to a freeze a few minutes later.
5 Likes
There may still be other options to try which better preserve the health of the battery.
I have similar problems however on an 11 gen with a Thunderbolt 3 dock. A reason for this seems to be that charging does not really stop when the charge limit is reached. The laptop is constantly switching between charging and discharging.
But I have not seen any disconnects since I started forcing the charging current to
zero with fw-ectool using this command:
sudo ectool chargecurrentlimit 0
If you don’t need the full bandwidth to your docking station you also may consider downgrading the connection. In my case it also helps to force the connection to USB 3 instead of Thunderbolt 3 by using an USB 3 cable.
1 Like
What does this do? I’m guessing it prevents the battery from charging, so effectively it’s running on external power.
I see the same with a CalDigit TS4, on a 12th generation, BIOS 3.04.
One way to reproduce this is (assuming battery level is >30% and <99%):
while true
date ; sudo ectool fwchargelimit 30 ; sudo ectool console | grep "Battery\|Charge Limit" ; sleep 60 ; date ; sudo ectool fwchargelimit 99 ; sudo ectool console | grep "Battery\|Charge Limit" ; sleep 10
end
The dock disconnects basically every time when issuing fwchargelimit 30. The sleeping time in between is necessary. When running watch ectool battery in parallel, and looking at the numbers, it seems that the disconnects are triggered by switching from high charging current to 0. (The sleeps ensure that some high enough charging current builds up before forcing it to 0 again.)
I’ve also once observed a disconnect in the other direction, i.e., when switching from 0 to high charging current.
I wrote a small daemon that works around this:
The idea is to use ectool chargecurrentlimit to set the charge current slowly to a low value when above 60%, and I’ve set the firmware charge limit in the BIOS to 65%. This makes sure that whenever I’m close to 65%, only a low charge current is applied.
I haven’t experienced any disconnects with that daemon.
This really appears to be a hardware issue. There are multiple threads about this, reproducing this with different generations, different BIOS versions, different OS, and different docks:
- [RESPONDED] 1TB expansion card disconnects randomly
- External display loses signal when charging via docking station
This is related to [TRACKING] Battery flipping between charging and discharging / Draws from battery even on AC . See my comments there.
4 Likes
I’d be lying if I said that I’ve been tracking it since I stopped responding, It wore me out trying to debug it so I haven’t dealt with it since the first 3 months. But this sounds like it has a great chance of being the issue, I will test it once I have replaced my display (stepped on it a couple months ago and decided to wait for the Matt displays before purchase). Mine has been set to charge to a max of 60% since day one for battery longevity. While proving that the issue is gone could take weeks since it didn’t happen reliably frequently for me, The fact you think you have identified the root cause and provided a script to reliably trigger it means that all the work for STR has been done and I just have to verify it! I must say I am very hopeful that my expansion bays don’t work.
And seeing the reason given for closing my thread, I would like to note that although I originally plan to run an OS on it, It didn’t work as a media bay either once I gave up on that. All the hours I spent testing it were done with the OS on an NVME an expansion card being used for media files or just as a drive to write and read random data from for testing. I knew it was a hardware issue isolated to the framework itself since the cards (I bought a second one just to verify the first one wasn’t the issue) worked perfectly fine in other systems. This would finally explain why, and we can move on to resolving the actual issue of true! (Oh hell, you even provided a temporary work around! Literally all the work! Pre-emptive thank you!)
Oh also note that I’ve had issues with my display port card the few times I’ve used it. Unlike with my storage card, I didn’t notice that much since the issue would resolve itself. A disconnects for a few milliseconds causing a display to flicker off and then back on required me to be actively looking at the screen. Storage card couldn’t resolve itself since the colonel would assign it a new name in the file system would remount the old version as read only. I have not, however, ever had an issue with the usba card… But I also never used it after initial install which I didn’t do plugged in.
I can’t remember if I ever tested anything while unplugged, It just takes so long for the issue to reliably appear without knowing the trigger that leaving it unplugged couldn’t actually prove the issue was no longer present before the battery died. Only thing I can do while I’m not plug is prove that it’s not a charging related issue since the other way around isn’t an option with occasional 4days between random disconnects. We will see… Now that there is hope and direction I will gladly test again. Hopefully this time I won’t go insane doing so!
On an off note, I had a 50-50 chance of my Wi-Fi disconnecting at the same time as the card stopped working. Turning wifi off and back on with the wifi/Bluetooth button resolved the issue. I could never figure out if they were related, and the issue happens without the card in as well So I was assuming it was a bad router… But perhaps the power issue extends beyond the expansion bay and may affect some other modules as well. But I’m getting ahead of myself here. We can come back around to explore the full scope of this issue after getting the community to test and hopefully prove the root cause with expansion cards. How many people have tried both your STR to prove it, and your work around thus far that you know about? And how many days or weeks or months have people gone without issue since implementing your work around?
@Matt_Hartley I have done 0 testing since my screen is broken ATM but I think this is probably it! Finally an explanation not just why my card works reliably with other computers but not the framework, but also an explanation on why there’s also a split in the community as to whether or not there’s a problem with cards with some even running distros off of it. Distros, running off of it, btw is normal, fine for usb. Completely should not be a problem. Something we kinda wanted to do. An what more something the product page itself advertises as a feature since what else would you need this speed for. I understand the thread was going nowhere, but that bull doesn’t pass as a response, it’s not 2005 anymore and your 1tb card supports it fine as advertised, the laptop however does not under some not yet understood conditions.
Most people don’t configure charge limits, if the vast majority of the people who had issues did… Everything starts to make sense. If you guys are able to reproduce the issue using the script he provided, You should probably also poll those who reported having issues and ask if they set the charging threshold limit. My knowledge you haven’t yet identify the root cause of the issue, and it’s way too frequent oven issue for those of us who have the issue on only your laptops to hand wavingly say you shouldn’t trust in USB… Again, it’s not 2005, and your expansion cards are not 5usd Walmart USB sticks
Off today, so will make this short.
Looks like someone above has a script to address this.
As stated in multiple other threads, please see this kb article, dock vs expansion card section.
Some docks have shown to be successful, others have had mixed results.
This remains unchanged.
This looks promising for those docks that present this problem and may offer some help here.
https://community.frame.work/t/thunderbolt-disconnections-on-arch-linux/32868/13?u=matt_hartley
1 Like
Hi @Neil_Green,
I think what @real_or_random said, already answers your question.
But you’re right, sudo ectool chargecurrentlimit 0 cuts off the charging current. I only use it, when connected to the dock, from which the laptop gets external power. The idea was actually to reduce the flipping between relatively high charge currents and discharging that I always observed shortly after disconnects. So I suspected this to be the cause of the problem.
If the battery is below the charging limit and I want to charge it I often use sudo ectool chargecurrentlimit 300 or other low limits instead. This extends the charging time. And I’ve never seen disconnects while charging. It also appears that the charge current limit does not need to be set to 0. A low charge current limit seems to be sufficient, even if the battery has reached the ‘fwchargelimit’.
Right, and that’s what my script does in fact. I edited the description above.
Just me so far, but I’ve been using it for a few weeks now.
Indeed. I’d still appreciate it if you could confirm the possibility that this is a hardware or a firmware problem, and forward it to whoever is responsible and can take a look. Here’s a possible explanation:
The logic in Blaming EmbeddedController/baseboard/fwk/battery.c at 72748b7d0aa4ac18204f00608ed61451b10314a3 · FrameworkComputer/EmbeddedController · GitHub (this code is custom to the Framework!) sets the charging mode to CHARGE_CONTROL_IDLE when the set charge threshold is reached. If the charge current happens to be high at this point, it’s suddenly reduced to 0, and this is precisely what causes disconnects in my experiments.
Note that CHARGE_CONTROL_IDLE is even set if the max threshold is 100% (i.e., the default), but still, it seems that only users with a threshold <100% seem to suffer from disconnects. I suspect that is because the other parts of the charging logic make sure that the charge current is low when close to 100%. And when a low current is suddenly set to 0, this does not result in a disconnect according to my experiments.
All of this is speculation until tested, but it seems plausible that this is the cause. A second reason to investigate this code further is that it may be the reason for the flipping between charging and discharging.
4 Likes
Thanks for this.
I finally got round to setting it up yesterday, and today’s result so far look very promising. I’ve set the charge limit to 60% and, with the appropriate tweak to the shell script, I’ve not seen any disconnects so far while the laptop is charged to the limit.
I would be happy to escalate to engineering. Do we have an active ticket open for this?
2 Likes
No. Would that be a strict requirement to escalate it? My write-up above has everything that I know about this issue.