Excellent, I appreciate the info! It’s funny, across here, Reddit, and the Ars Technica article comments, you’ve been the one keeping level-headed and providing good, solid info. I really appreciate that.
1 Like
I try to keep the anger down when writing things and remain fair and also acknowledge positive developments etc. In the hopes of getting Framework to change their stance and rethink things. And getting me the products I would like to have. Although I have been getting a bit more snarky over time.
Maybe we get lucky and the same or other authors actually pick up on the 3.08b release situation and how FW is again trying to keep that quiet.
Maybe that gets us even more transparency and hope for the future.
4 Likes
However, now with Linux, entering “deep” suspend (in the /sys/power/mem_sleep sense) consistently soft bricks the device. Once in this state, a mainboard reset is required.
Funny because I updated the bios last week to 3.06beta, and the issue you report is exactly what lead me to this thread. The poweroff is not working either.
I’ll contact support. My 2 rear USB ports are dead (except for charging) and this suspend issue are a blocker for me.
1 Like
OK, I tried the update to 3.08, and it completely broke my motherboard. The 4 usb ports are not visible even in the BIOS (boot selector) anymore. Contacting support.
3 Likes
Quick update - 3.08b update completed, Retimer issue doesn’t appear to have hit as all USB ports appear to be working.
1 Like
Sorry to stumble upon the flowers on the carpet, as we say in french, but did you really mean the 12th gen 3.19 update? I thought that was 3.08?
Also, you seem to be saying we can test the update now, is there an actual way to do the 3.08 upgrade now on Linux? (And no, install windows is not a way.)
Thanks.
2 Likes
Also, it seems relevant to mention there was a whole blog post about this very topic on the website now:
I found it’s pretty thin on actual contents. The only new thing in there is:
for Linux we’re developing a new updater to handle the specific firmware regions involved.
which, quite frankly, is actually pretty disappointing. I did the 3.06b update on Debian Linux through LVFS and it Just Worked. I understand this update is trickier, but I find it regrettable that we’re building a whole new way to update firmware instead of leveraging what the community has been working really hard on for years at this point, and that has been adopted by many large, much more commercial/proprietary vendors than Framework, including Dell, HP, Lenovo…
Really, I would love to hear more about what the actual problem is with LVFS and whether or not the LVFS people have to say about this… This really sounds an awful lot like what Purism and System76 ended up with, which is that they’ve basically given up on LVFS. I didn’t buy it from them, and I don’t buy it from Framework either.
Surely I must be missing something.
4 Likes
not to go far off topic here, but yeah, americans do love it, and seem to have rarely heard it. It’s pretty colloquial (in french!) where I’m from, but maybe not in France…
6 Likes
Same here. @pixelforest don’t reinvent the wheel, or have a case of “not designed here” let’s design something that in the long term is just one more thing we will support until we have to drop it…mir, upstart, unity same concept different projects.
2 Likes
Having unpacked both installers and run checksums on the actual firmware files: the firmware capsules / images themselves are identical. The only other thing that changed are the .cat files, which are just signatures etc. for Windows. Which is probably caused by how they automate packing the installer, that those are regenerated when building a new one.
I am curious: what driver could impact your Windows firmware updater? Only the ME Update runs from inside Windows, no?
Also, your driver bundles do not really have a name, only the date published on the release website (that we cannot see in Windows after the fact), because it is just a bundle of various other installers plus remapping the fn+F12 key to a webpage.
To my knowledge, there also has only ever been one driver bundle for 12th gen.
So the answer to this question can only by “i don’t know” or “the only driver bundle you ever had”. If particular drivers impact the parts of the update that run in Windows wouldn’t it be better to ask for those versions? Because we can see for example the currently installed Intel chipset drivers version easily. And other drivers may very well have been updated above what is in your driver bundle (iGPU, WiFi, BT).
Now this is a really curious point. The EFI installer you have released includes the ME firmware file (binarily identical to the one that is actually installed in the Windows installer, 10MiB) and the ME updater.
Are you saying you packed both of those in there, but it is never actually executed? Then, saying that would have probably been a good explanation for why you pulled that updater down again.
On the other hand, FWUpdLcl.efi looks to be a EFI commandline program, just like the other updaters that are run from your EFI Shell script. That would not look to be hard to run in an EFI shell, that you are already using, just like how it is run separately and before the other firmware updates are installed after the reboot with the Windows installer.
Or are there deeper problems, and the official Intel installer you packed in there would fail to run, if it ever were executed?
What about the capsule update process you have been using from Windows and Linux/LVFS. Is that sth. Framework is planning on improving to the point where Framework can simply also ship ME updates or the special logic for standalone boards through the capsules and the OS agnostic updater that applies those updates?
Or will there only ever be a Windows and a standalone, bootable update process and no more Linux/LVFS updates for everything that has ME firmware? Do you have control over the Capsule updater and could enhance that to also update ME firmware (similar to how other manufacturers update everything through that process, precisely because it is OS agnostic)?
5 Likes
After a tiny bit of reverse engineering: Yes, the structure of the EFI updater is as I expected.
- It boots the shell and if not aborted will execute the startup.nsh script, that will simply execute “fwupdate.efi --allupdate”
- you can hit escape and operate everything from the shell manually
- fwupdate.efi calls itself “framework_tool: Swiss army knife for Framework laptops”
- it can read versions, read charging state (in detail, it distinguishes charger, connected, battery state and battery charging) check header information of capsule files.
- retimer updates in EFI installer are valid capsules, the main firmware is not
- fwupdate.efi seems to execute the framework-firmware-update\res\firmware.nsh script with arguments like “[path_to_firmware folder] pd flash-left” which will actually call the other efi executables that do the actual updates, like H2OFFT.efi, CyPDADLt1.efi. There seem to be no obvious references to the ME update in there
- FWUpdLcl.efi is the official Intel ME Updater. It executes, it reports the same program version as the Windows variant that is used as part of the Windows update process. It can read the current ME firmware version, the version of the ME update and seems completely capable of running the update. It just looks like it is never called by the fwupdate.efi that manages all of it. But executing it manually on the shell would work.
- By the features fwupdate.efi has, it looks like it is intended to read all current versions and critical settings to determine what needs to be updated, in what order etc. And it only has a placeholder to determine the current ME firmware version. And so it does nothing with that.
- But when you consider that the Windows updater simply executes unconditionally FWUpdLcl.exe with “-allowsv”, which forces an update of the ME firmware every time the installer is launched, even if the firmware is already the same, it is not like the Windows installer does a whole lot of sanity checks before it simply hands over to Intel’s updater to apply the update. If the other Retimer and BIOS updates where to fail or not complete after the ME update, the versions would be desynchronized with the officially released Windows updater.
- An actual, technical explanation of why the EFI installer cannot just launch the ME updater in the same way would be nice, to understand what about that is actually holding Framework up for over half a year, when it is not the case, that you just did not have the time to look at it at all.
- Also adds to the list of things that you need to explain: how did you manage to ever publish that updater if you find that is a critical problem. Clearly you packaged the ME updater and firmware in it with some kind of plan. And why would you not tell beta testers that already have used the EFI Updater before it was depublished that their update is incomplete or how it can be completed. If unmatched ME and BIOS versions were a problem, you’d need to tell people that they are in a unsupported state and how to recover. Even if you’d have to recommend the Windows installer for that, as you are already doing.
Note: I have not actually executed any updates with the pulled EFI updater, only confirmed that it can access and for example dump the current ME firmware. And if fwupdate.efi would internally construct the path to FWUpdLcl.efi from single characters, instead of hardcoding the full path as for all the other binaries and executables, I would have missed that.
But I’d consider @pixelforest post as confirmation, that the ME Updater is actually never called, even though it looks to be just a single commandline to by executed on the EFI shell to fix that.
I’d be curious if any of the customers that have used the EFI updater as it was originally intended actually has the new ME firmware on their Framework or if it actually stayed the same.
Using the EFI updater, aborting startup.nsh (you could also edit the “–allupdate” argument out of it to defuse it from auto-executing actual changes to firmware).
Once on the shell, executing “framework-firmware-update\firmware\FWUpdLcl.efi -fwver” will read out the currently installed ME version. This is an alternative to ways to get that version from within Linux or from the BIOS itself. I am not recommending people to actually use that to update ME, before Framework confirms anything or you want to risk your device. I am just trying to understand where Framework stands and what they plan and need to do, if they are not able to simply tell us after making many posts about wanting to improve communication.
And I am also trying to fact-check the non-technical and vague things they claim.
6 Likes
The main page they mention Intel CSME 16.0.15.1810v3.1
while under “enhancements” they mention: “Update Intel CSME firmware to 16.1.30.2269v4_A0_Corporate”
I just checked in my bios, (ive ran the EFI update, like you concluded, Intel ME dint get updated)
Im on Intel ME 16.0.15.1810 / CORPORATE
InsydeH20 Bios is HFW30.03.08
(I rechecked the video I took of the update process, no mention of intel ME engine ever checked or updated)
1 Like
Thank you for the answer. After using the Windows updater / manually installing the different firmwares from Windows, BIOS shows “16.1.30.2269 / Corporate”.
ME-Updater from EFI Shell or Windows would show:
The last command is the actual command to run the update. And it aborts on my device because that is the version that is already on it and I did not want to risk actually forcing it to run just yet.
Edit:
So without an actual answer and explanation from Framework, from this I conclude that
Note: We have removed the EFI update for now until we improve the stability of this update method based on user feedback below.
is a lie or at least obfuscation. And the actual & more important reason they do not want to admit to, is that they produced a half-baked updater or simply failed to communicate the instructions to run all of the update internally. Or the update process would actually brick the device. In which case I find it suboptimal to ever publish it anywhere and leave the EFI installer still hosted and not warn of it.
And since it looks to be the official updater from Intel, any problems with using it would most likely be a platform problem. I.e. a fault already in the BIOS that confuses the updater or makes it incompatible with what Intel defines as correct behavior.
Edit2: the backup dump the ME updater can create is identical to the one the officially released updater for Windows makes. If the updater does not fail somehow due to a privilege problem (i.e. the ME firmware is not writeable / modifiable for some reason. Would fail before it has a chance to make changes to the system), I find it hard to believe that it would brick. But the clear risk remains.
Edit3: confirmed it runs perfectly. Past users have already found that manual ME update and successfully ran it. New installer release uses the same exact updater with same arguments.
2 Likes
you command seemed to indeed run fine. and confirmed the installed ME version.
So it seems EFI can update all, but they disabled the CSME portion (for some reason), Im not sure why Id like to update the ME section tho. would this be the cause of some PCI stuff not working like when “they” disabled ME on other older platforms? Feedback from FW would be nice. (I prever a secure main Firmware and skip all this stuff and just get Coreboot… I dont mind a weird not fancy gui to do stuff) offtopic.
That backup, binwalk is not seeing much. compressed?
ME is deeply involved in Intel’s stuff. It may do certain system management actions. It provides a lot of functionality, like the firmware TPM (that FW does not use as far as I see), DRM support for Windows. ARC GPUs are updated through it.
It is probably involved in managing standby modes on some level.
It has been encrypted forever, because it is part of the chain of trust for Intel. ME can undermine every piece of security the processor or OS otherwise try to provide. And ME has functionality like remote management and code execution (like AMT with running and orchestrating virtual machines running in the background scanning for problems, attaching virtual CD drives to remote install an OS or do maintenance or decouple an infected system from the corporate network).
So most of the reasons I have seen for ME updates is fixing security issues in there (I believe it is internally based on some embedded Linux. At least it was in the past. Just with vast access to the system. Also the reason why people want to disable it. It is propriatary code that cannot be audited). But because the BIOS needs to interact with it during bring-up they typically want a specific BIOS version paired with a specific ME version to exclude any compatibility issues with untested combinations. I have no idea how often ME updates might introduce breaking changes in its API, so that running unsupported combinations of ME and BIOS could cause instability as ME does not behave how BIOS expects. The people building their own EFI installers for 11th gen have explicitly not been doing the ME update (lack of updater, they are reusing an older EFI installer without the ME updater). And I have seen no reports of sth. that I would suspect to be caused by non-matched ME versions. But the possibility cannot be excluded.
2 Likes
And Intel Bootguard sadly (no idea if this emediatly influencing ME) but thats turned on on these machines. This might be the very last Intel/AMD I buy. Alott we dont know for sure with current gen Intel ME, previous ones where indeed definitly alott involved or could potentially have. Disabling it (as much possible) while maintaining normal operations would be preferred.
I have not looked deeply into it. But as I understand Bootguard is just Intel’s marketing name for authorized boot and their root of trust.
A manufacturer fuses their signing / encryption keys in there. The processor will only boot code signed to match. So you theoretically cannot undermine security of SecureBoot / BIOS etc. by flashing a different BIOS that skips security.
As long as you do not find exploits in the BIOS itself.
Basically Intel’s version of a locked Android bootloader. Only that the bootloader we are talking about is part of the platform. And the manufacturers BIOS is loaded by it. You want that unless you are a developer of BIOS or want to self-sign everything.
AMD will have an equivalent for the exact same reasons.
Edit:
But yeah, that kind of thing will prevent you from maintaining yourself everything it “authorizes” / authenticates as a result. But not having anything means anybody with access to the platform could place a BIOS level rootkit into your platform that you won’t be able to see. And they would not even need to rely on any exploit.
Edit2: without sth. like it, there’d be almost no reason to fix BIOS security issues like LogoFail, because it’d be harder to exploit them than just updating the firmware directly.
Update: I’m realizing just now that my right USB are only working in USB 2.1 now.
I’m still stuck on code 10 (failed to update firmware) and I can’t do anything.
I want to remind that when I initially tried to update I only ever had one progress bar in the whole process.
- FAILURE SKU# and SYS SERIAL NUMBER: unsure
- SYS CONFIG: 12th Gen Intel Core i7-1260P
- RAM: 2x Crucial 32 GB DDR4-3200 SODIMM 1.2V CL22
- SSD: Samsung SSD 980 PRO 2TB
- Wi-Fi: AX210NGW
- External Devices/Other: Unplugged everything while rebooting to do the update.
- EXPANSION CARD TYPES: 2x USB-C, 2x USB-A
- BIOS VERSION: 3.05
- DRIVER PACKAGE VERSION: Well I don’t think there was any update so I don’t really know.
- OS VERSION: Windows 11
- FAIL RATIO: 100%
- STEP TO REPRODUCE: Windows package
Step 1 - Running msi
Step 2 - Reboot and unplugging everything from the USB cards - OBSERVED RESULT: Well nothing, it didn’t actually update (said failed in device manager).
- EXPECTED RESULT: Obviously this one, we expect it to flash successfully and be reflected in BIOS settings.
3 Likes