12th Gen Intel Core BIOS 3.08 Release

However, now with Linux, entering “deep” suspend (in the /sys/power/mem_sleep sense) consistently soft bricks the device. Once in this state, a mainboard reset is required.

Funny because I updated the bios last week to 3.06beta, and the issue you report is exactly what lead me to this thread. The poweroff is not working either.

I’ll contact support. My 2 rear USB ports are dead (except for charging) and this suspend issue are a blocker for me.

1 Like

OK, I tried the update to 3.08, and it completely broke my motherboard. The 4 usb ports are not visible even in the BIOS (boot selector) anymore. Contacting support.

3 Likes

Quick update - 3.08b update completed, Retimer issue doesn’t appear to have hit as all USB ports appear to be working.

1 Like

Hey everyone!

Thank you for all of your feedback and we are working on a stable Linux update method via EFI (among other improvements you all have noted in the thread for our next release) as quickly and as high of quality as we can. We also read all your feedback (I have personally read every post here and tried to reproduce many of the issues!), so keep it coming!

We know words are cheap, and you will only believe us once the work is completed and delivered to all of you.

In the meantime, some more context with the missing EFI updater on the current 11th and 12th Gen BIOS releases (3.19 and 3.08 respectively) we thought would be helpful.

The reason we haven’t released the EFI update is that the EFI updater is missing functionality to update the CSME region to the latest version which includes security updates from Intel. The EFI updater we currently use is based on work from Framework, which did not originally account for the possibility of CSME updates. We’re currently developing a new update script, which allows us to update this region going forward.
Again, we don’t want to make promises or empty words in forum posts, but we are working to improve the communication frequency and look forward to deliver more updates for all of you over time.

In the meantime, when you have an issue it will help us move significantly faster on our side to reproduce and work with our partners if you can use the bug report template (as much information as possible helps us set up repro cases to work towards a root cause and solution!)

We very much need the following to track down where things are failing OR succeeding (as much information as you can, especially the reproduction steps)

  • FAILURE SKU# and SYS SERIAL NUMBER: (Use this guide to get access to the mainboard )
  • SYS CONFIG: (i7, i5, specific model details, get the details from the motherboard sticker, same location as above)
  • RAM: Brand and how much, 1 or 2 sticks
  • SSD: Brand/model and how large is the capacity. If removed, please indicate.
  • Wi-Fi: Which wifi card? If removed, please indicate.
  • External Devices/Other: Anything attached? If so, what and how? If not, please indicate.
  • EXPANSION CARD TYPES: What cards were inserted? Example: 2x Display Port, 2x USB-A
  • BIOS VERSION: Which BIOS version were on on before attempting the flash?
  • DRIVER PACKAGE VERSION: If known and if using Windows.
  • OS VERSION: if Windows, 10 or 11? If Linux, which distro and release version? For Arch, this would be Arch, fully updated or not, please indicate.
  • FAIL RATIO: Can you reliably reproduce this issue 100% of the time? Is it only 50% of the time? Occurs randomly?
  • STEP TO REPRODUCE: EFI Shell via USB drive, Windows package, and the steps you recall taking. Just do your best. I realize no one will remember all of this.
    Step 1 -
    Step 2 -
    Step 3 -
    Etc, Etc.
  • OBSERVED RESULT: How it failed, what you saw on the screen.
  • EXPECTED RESULT: Obviously this one, we expect it to flash successfully and be reflected in BIOS settings.
  • ISSUE RECOVERY METHOD: If you were able to recover, walk us through this process.
  • EXTERNAL DEVICE MODE or NAME: For USB flash drives, Generic or the brand.
21 Likes

Sorry to stumble upon the flowers on the carpet, as we say in french, but did you really mean the 12th gen 3.19 update? I thought that was 3.08?

Also, you seem to be saying we can test the update now, is there an actual way to do the 3.08 upgrade now on Linux? (And no, install windows is not a way.)

Thanks.

2 Likes

Also, it seems relevant to mention there was a whole blog post about this very topic on the website now:

I found it’s pretty thin on actual contents. The only new thing in there is:

for Linux we’re developing a new updater to handle the specific firmware regions involved.

which, quite frankly, is actually pretty disappointing. I did the 3.06b update on Debian Linux through LVFS and it Just Worked. I understand this update is trickier, but I find it regrettable that we’re building a whole new way to update firmware instead of leveraging what the community has been working really hard on for years at this point, and that has been adopted by many large, much more commercial/proprietary vendors than Framework, including Dell, HP, Lenovo…

Really, I would love to hear more about what the actual problem is with LVFS and whether or not the LVFS people have to say about this… This really sounds an awful lot like what Purism and System76 ended up with, which is that they’ve basically given up on LVFS. I didn’t buy it from them, and I don’t buy it from Framework either.

Surely I must be missing something.

4 Likes

Sorry to stumble upon the flowers on the carpet, as we say in french, but did you really mean the 12th gen 3.19 update? I thought that was 3.08?

Hey @anarcat - good catch, that was a typo and I modified it in the above post.

you seem to be saying we can test the update now, is there an actual way to do the 3.08 upgrade now on Linux?

Unfortunately, the way I have worked around it is using the Windows BIOS installer.

Yes, installing Windows, running the BIOS update and then wiping the Windows partition afterwards…not an acceptable solution as you mentioned, but it got me there in the meantime.

I’m also working with @Kieran_Levin to see if we have another solution we can write a guide on (non-Windows) in the meantime, no promises, but I will keep everyone posted.

Edit: PS - I love this saying, have never heard it as an American, may be using it more in my own day to day :stuck_out_tongue:

stumble upon the flowers on the carpet

5 Likes

not to go far off topic here, but yeah, americans do love it, and seem to have rarely heard it. It’s pretty colloquial (in french!) where I’m from, but maybe not in France…

6 Likes

Same here. @pixelforest don’t reinvent the wheel, or have a case of “not designed here” let’s design something that in the long term is just one more thing we will support until we have to drop it…mir, upstart, unity same concept different projects.

2 Likes

and @anarcat

According to this, did your CSME firmware update? 3.08b is just the msi update or no?

Having unpacked both installers and run checksums on the actual firmware files: the firmware capsules / images themselves are identical. The only other thing that changed are the .cat files, which are just signatures etc. for Windows. Which is probably caused by how they automate packing the installer, that those are regenerated when building a new one.

I am curious: what driver could impact your Windows firmware updater? Only the ME Update runs from inside Windows, no?
Also, your driver bundles do not really have a name, only the date published on the release website (that we cannot see in Windows after the fact), because it is just a bundle of various other installers plus remapping the fn+F12 key to a webpage.
To my knowledge, there also has only ever been one driver bundle for 12th gen.
So the answer to this question can only by “i don’t know” or “the only driver bundle you ever had”. If particular drivers impact the parts of the update that run in Windows wouldn’t it be better to ask for those versions? Because we can see for example the currently installed Intel chipset drivers version easily. And other drivers may very well have been updated above what is in your driver bundle (iGPU, WiFi, BT).

Now this is a really curious point. The EFI installer you have released includes the ME firmware file (binarily identical to the one that is actually installed in the Windows installer, 10MiB) and the ME updater.
Are you saying you packed both of those in there, but it is never actually executed? Then, saying that would have probably been a good explanation for why you pulled that updater down again.

On the other hand, FWUpdLcl.efi looks to be a EFI commandline program, just like the other updaters that are run from your EFI Shell script. That would not look to be hard to run in an EFI shell, that you are already using, just like how it is run separately and before the other firmware updates are installed after the reboot with the Windows installer.
Or are there deeper problems, and the official Intel installer you packed in there would fail to run, if it ever were executed?

What about the capsule update process you have been using from Windows and Linux/LVFS. Is that sth. Framework is planning on improving to the point where Framework can simply also ship ME updates or the special logic for standalone boards through the capsules and the OS agnostic updater that applies those updates?
Or will there only ever be a Windows and a standalone, bootable update process and no more Linux/LVFS updates for everything that has ME firmware? Do you have control over the Capsule updater and could enhance that to also update ME firmware (similar to how other manufacturers update everything through that process, precisely because it is OS agnostic)?

5 Likes

After a tiny bit of reverse engineering: Yes, the structure of the EFI updater is as I expected.

  • It boots the shell and if not aborted will execute the startup.nsh script, that will simply execute “fwupdate.efi --allupdate”
  • you can hit escape and operate everything from the shell manually
  • fwupdate.efi calls itself “framework_tool: Swiss army knife for Framework laptops”
    • it can read versions, read charging state (in detail, it distinguishes charger, connected, battery state and battery charging) check header information of capsule files.
    • retimer updates in EFI installer are valid capsules, the main firmware is not
  • fwupdate.efi seems to execute the framework-firmware-update\res\firmware.nsh script with arguments like “[path_to_firmware folder] pd flash-left” which will actually call the other efi executables that do the actual updates, like H2OFFT.efi, CyPDADLt1.efi. There seem to be no obvious references to the ME update in there
  • FWUpdLcl.efi is the official Intel ME Updater. It executes, it reports the same program version as the Windows variant that is used as part of the Windows update process. It can read the current ME firmware version, the version of the ME update and seems completely capable of running the update. It just looks like it is never called by the fwupdate.efi that manages all of it. But executing it manually on the shell would work.
  • By the features fwupdate.efi has, it looks like it is intended to read all current versions and critical settings to determine what needs to be updated, in what order etc. And it only has a placeholder to determine the current ME firmware version. And so it does nothing with that.
  • But when you consider that the Windows updater simply executes unconditionally FWUpdLcl.exe with “-allowsv”, which forces an update of the ME firmware every time the installer is launched, even if the firmware is already the same, it is not like the Windows installer does a whole lot of sanity checks before it simply hands over to Intel’s updater to apply the update. If the other Retimer and BIOS updates where to fail or not complete after the ME update, the versions would be desynchronized with the officially released Windows updater.
  • An actual, technical explanation of why the EFI installer cannot just launch the ME updater in the same way would be nice, to understand what about that is actually holding Framework up for over half a year, when it is not the case, that you just did not have the time to look at it at all.
  • Also adds to the list of things that you need to explain: how did you manage to ever publish that updater if you find that is a critical problem. Clearly you packaged the ME updater and firmware in it with some kind of plan. And why would you not tell beta testers that already have used the EFI Updater before it was depublished that their update is incomplete or how it can be completed. If unmatched ME and BIOS versions were a problem, you’d need to tell people that they are in a unsupported state and how to recover. Even if you’d have to recommend the Windows installer for that, as you are already doing.

Note: I have not actually executed any updates with the pulled EFI updater, only confirmed that it can access and for example dump the current ME firmware. And if fwupdate.efi would internally construct the path to FWUpdLcl.efi from single characters, instead of hardcoding the full path as for all the other binaries and executables, I would have missed that.
But I’d consider @pixelforest post as confirmation, that the ME Updater is actually never called, even though it looks to be just a single commandline to by executed on the EFI shell to fix that.

I’d be curious if any of the customers that have used the EFI updater as it was originally intended actually has the new ME firmware on their Framework or if it actually stayed the same.
Using the EFI updater, aborting startup.nsh (you could also edit the “–allupdate” argument out of it to defuse it from auto-executing actual changes to firmware).
Once on the shell, executing “framework-firmware-update\firmware\FWUpdLcl.efi -fwver” will read out the currently installed ME version. This is an alternative to ways to get that version from within Linux or from the BIOS itself. I am not recommending people to actually use that to update ME, before Framework confirms anything or you want to risk your device. I am just trying to understand where Framework stands and what they plan and need to do, if they are not able to simply tell us after making many posts about wanting to improve communication.
And I am also trying to fact-check the non-technical and vague things they claim.

6 Likes

The main page they mention Intel CSME 16.0.15.1810v3.1
while under “enhancements” they mention: “Update Intel CSME firmware to 16.1.30.2269v4_A0_Corporate”

I just checked in my bios, (ive ran the EFI update, like you concluded, Intel ME dint get updated)
Im on Intel ME 16.0.15.1810 / CORPORATE

InsydeH20 Bios is HFW30.03.08
(I rechecked the video I took of the update process, no mention of intel ME engine ever checked or updated)

1 Like

Thank you for the answer. After using the Windows updater / manually installing the different firmwares from Windows, BIOS shows “16.1.30.2269 / Corporate”.
ME-Updater from EFI Shell or Windows would show:

The last command is the actual command to run the update. And it aborts on my device because that is the version that is already on it and I did not want to risk actually forcing it to run just yet.

Edit:
So without an actual answer and explanation from Framework, from this I conclude that

Note: We have removed the EFI update for now until we improve the stability of this update method based on user feedback below.

is a lie or at least obfuscation. And the actual & more important reason they do not want to admit to, is that they produced a half-baked updater or simply failed to communicate the instructions to run all of the update internally. Or the update process would actually brick the device. In which case I find it suboptimal to ever publish it anywhere and leave the EFI installer still hosted and not warn of it.
And since it looks to be the official updater from Intel, any problems with using it would most likely be a platform problem. I.e. a fault already in the BIOS that confuses the updater or makes it incompatible with what Intel defines as correct behavior.

Edit2: the backup dump the ME updater can create is identical to the one the officially released updater for Windows makes. If the updater does not fail somehow due to a privilege problem (i.e. the ME firmware is not writeable / modifiable for some reason. Would fail before it has a chance to make changes to the system), I find it hard to believe that it would brick. But the clear risk remains.

Edit3: confirmed it runs perfectly. Past users have already found that manual ME update and successfully ran it. New installer release uses the same exact updater with same arguments.

2 Likes

2024-04-18-173738_4176x1504_scrot
you command seemed to indeed run fine. and confirmed the installed ME version.

So it seems EFI can update all, but they disabled the CSME portion (for some reason), Im not sure why Id like to update the ME section tho. would this be the cause of some PCI stuff not working like when “they” disabled ME on other older platforms? Feedback from FW would be nice. (I prever a secure main Firmware and skip all this stuff and just get Coreboot… I dont mind a weird not fancy gui to do stuff) offtopic.

That backup, binwalk is not seeing much. compressed?

ME is deeply involved in Intel’s stuff. It may do certain system management actions. It provides a lot of functionality, like the firmware TPM (that FW does not use as far as I see), DRM support for Windows. ARC GPUs are updated through it.
It is probably involved in managing standby modes on some level.

It has been encrypted forever, because it is part of the chain of trust for Intel. ME can undermine every piece of security the processor or OS otherwise try to provide. And ME has functionality like remote management and code execution (like AMT with running and orchestrating virtual machines running in the background scanning for problems, attaching virtual CD drives to remote install an OS or do maintenance or decouple an infected system from the corporate network).

So most of the reasons I have seen for ME updates is fixing security issues in there (I believe it is internally based on some embedded Linux. At least it was in the past. Just with vast access to the system. Also the reason why people want to disable it. It is propriatary code that cannot be audited). But because the BIOS needs to interact with it during bring-up they typically want a specific BIOS version paired with a specific ME version to exclude any compatibility issues with untested combinations. I have no idea how often ME updates might introduce breaking changes in its API, so that running unsupported combinations of ME and BIOS could cause instability as ME does not behave how BIOS expects. The people building their own EFI installers for 11th gen have explicitly not been doing the ME update (lack of updater, they are reusing an older EFI installer without the ME updater). And I have seen no reports of sth. that I would suspect to be caused by non-matched ME versions. But the possibility cannot be excluded.

2 Likes

And Intel Bootguard sadly (no idea if this emediatly influencing ME) but thats turned on on these machines. This might be the very last Intel/AMD I buy. Alott we dont know for sure with current gen Intel ME, previous ones where indeed definitly alott involved or could potentially have. Disabling it (as much possible) while maintaining normal operations would be preferred.

I have not looked deeply into it. But as I understand Bootguard is just Intel’s marketing name for authorized boot and their root of trust.
A manufacturer fuses their signing / encryption keys in there. The processor will only boot code signed to match. So you theoretically cannot undermine security of SecureBoot / BIOS etc. by flashing a different BIOS that skips security.
As long as you do not find exploits in the BIOS itself.
Basically Intel’s version of a locked Android bootloader. Only that the bootloader we are talking about is part of the platform. And the manufacturers BIOS is loaded by it. You want that unless you are a developer of BIOS or want to self-sign everything.
AMD will have an equivalent for the exact same reasons.

Edit:
But yeah, that kind of thing will prevent you from maintaining yourself everything it “authorizes” / authenticates as a result. But not having anything means anybody with access to the platform could place a BIOS level rootkit into your platform that you won’t be able to see. And they would not even need to rely on any exploit.

Edit2: without sth. like it, there’d be almost no reason to fix BIOS security issues like LogoFail, because it’d be harder to exploit them than just updating the firmware directly.

This is definitely our goal @nadb , and as you know, with quite the small internal team we like to be as efficient and leverage existing solutions wherever possible. LVFS would definitely be great for all updates.

5 Likes