Exploring the Embedded Controller

Whoops, you’re totally right. That printout is from the 11th gen Intel Framework Laptop; it shares an embedded
controller lineage with the rest of the Intel devices that are available today and may have some slightly different
features, sensors and reports from the AMD ones.

I haven’t seen any indication of battery temperature reporting on my AMD Frameworks Laptop.

Unfortunately, the ectool project requires ClangCL. It’s an optional component that ships with Visual Studio:

image

The module is probably not loading because it doesn’t have the required patches (which just landed for linux-next!)

If you can wait just a bit longer, it should start showing up in distribution kernels. Any version of ectool will work after that.

If you can’t: you can always disable secure boot and use a binary build of
ectool:

3 Likes

Nah, it’s not loading because it’s not signed. :slight_smile: Also, that patch seems to be for AMD laptops support, i have a 12th gen Intel…

Where do you get ectool? It doesn’t seem to be packaged in Debian, and I’m not a fan of “disable [security] and [run arbitrary binaries from GitHub]”… :slight_smile: Here (Debian stable), there seems to be a ectool program in the coreboot-utils package, but it’s orphaned so i guess that explains a lot… it certainly doesn’t have ectool temps for example:

anarcat@angela:~$ ectool temps all
Error: Extra parameter found.
usage: ectool [-vh?Vidq] [-w 0x<addr> -z 0x<data>]

   -v | --version:                   print the version
   -h | --help:                      print this help

   -V | --verbose:                   print debug information
   -p | --getports:                  get EC data & cmd ports from /proc/ioports
   -d | --dump:                      print RAM
   -i | --idx:                       print IDX RAM & RAM
   -q | --query:                     print query byte
   -w <addr in hex>                  write to addr
   -z <data in hex>                  write to data

So I guess I should update the debian package somehow! :slight_smile:

Oh! Sorry, I’m used to all new inquiries being about the AMD devices. In that case… I believe a newer version of the kernel package (working its way through sid) now comes with a signed cros_ec_lpcs.ko. :slight_smile:

Honestly, I used to get it by building upstream. You can also build Framework’s version. Both of these will require you to make util or make utils (I’ve forgotten which.)

It’s part of the overall EC source code. That’s why I pulled it out to live in a standalone repository! It doesn’t need the rest of the EC source code.

2 Likes

I found that upon $ sudo ectool battery
The “Desired current 2740 mA” is not a constant value, sometimes it gets higher sometimes lower, does that have anyting to do with temperature?

Edit: It goes to 3915 mA during discharging

the fwchargelimit can be set to 255, is it a bug?

1 Like

Hi, this is an amazing work! I am just struggling with the driver signing on Windows since I do not want to allow test signing and disable secure boot.

I thought about signing the Windows driver myself with a certificate, but just noticed that Microsoft changed their driver signing policies since Windows 10 1607 and only Microsoft-signed drivers via the Microsoft portal are now permitted? Sounds complicated.

Do you know if one can use the generic non-Framework EC software utilities that are already signed or somehow work around the signing, such as RWeverything or nbfc’s ec-probe? When I use ec-probe.exe monitor or ec-probe.exe dump, it gives me some memory dumps where some values change over time, but I have no idea if they’re meaningful and what addresses to look at in the AMD Framework 13 since all your old blog posts are about the Intel variants.

Note: Apparently the RWeverything driver is on Microsoft’s blocklist, but can be allowed by disabling the “Microsoft Vulnerable Driver Blocklist” feature.

Edit: I think that using ec-probe.exe dump stopped Framework from reporting battery status to Windows, it got stuck at 85% for a long time. Reboot fixed it.

If you would consider disabling the signed-driver blocklist, and installing RWeverything, you really should consider disabling Secure Boot. Vulnerable drivers are one of the main ways that the malware, which Secure Boot is supposed to stop, work. So, you know, just don’t install malware :wink: See also How a Microsoft blunder opened millions of PCs to potent malware attacks | Ars Technica (the blocklist wasn’t actually being updated for a couple years … people outside microsoft figured it out because active malware continued to bundle and use known vulnerable signed drivers).

EDIT: I’m cavalier about secure-boot because I used desktops/laptops heavily for over a decade while secure-boot didn’t exist yet, and it makes lots of interesting tinkering things painful/annoying.

I understand what you’re saying, but I still think we should figure out a reasonable way to sign the driver since more and more people are using it. :slight_smile:

It is not just about disabling secure boot, it is the fact that you are essentially installing a random unsigned driver from the internet. Even if I trust DHowett, someone can steal his password, login into his repository, and replace the driver by infected executables, all under his account. You are then going to install a virus and it could take weeks before anyone realizes.

Unfortunately, I do not really have any experience with the Microsoft procedures. The way I understand it, you need an EV certificate (easy to obtain, but quite expensive), register for Microsoft Entra ID and Microsoft Windows Hardware Developer Program (looks like you “just” sign a bunch of agreements, but it should be free), then the driver needs to pass the HLK tests (no idea about this, but it seems that Microsoft publishes VHLK virtual machines with everything set up to run the tests), and finally you upload the test results to the Microsoft portal, where it gets somehow approved and signed by Microsoft.

Sounds very elaborate. I dunno if you have to pay anything to Microsoft, but it seems the only paid part is the EV certificate, which is unfortunately quite pricy. The whole procedure sounds somewhat straightforward, but it could also be a big pain in the butt. I cannot judge :slight_smile:

I would like to help but not sure how. We have EV certificates in our company that we use to sign our software, but I am not sure I’d be able to push through the idea that we’d be signing this driver ourselves. Ideally, since Framework is also using it in their own scripts, Framework would sign the driver using their own certificates.

Hmm, maybe there is an easier solution that bypasses HLK, but an EV certificate is still required. The solution is called attestation signing (info 1, info 2), which signs the driver by Microsoft without performing the HLK tests. Such drivers cannot be published via Windows Update (which no one cares here), but they are still signed.

Anyway, I wanted to start this discussion mainly to understand if there is even any demand for getting it signed (except coming from me), and I mainly reacted to DHowett’s older message, in which he wrote that he was thinking about signing the driver.

Oh man. Great intro and overview over EC stuff! After having gone through a lot of the hx30 code, there would be so much opportunity to implement/tune a few features I’d like:
auto keyboard-backlight, keyboad-backlight timeout, keyboard brightness animation, get me back all the features I miss from my Dell XPS. Also, tune fan curve to oscillate less audibly near idle, more obvious visual indication of long-press power button interactions.
Most of that seems actually rather achievable if the EC is not already at its limit of handling realtime tasks.

But boy do I not want to brick my FW13 (12th gen) and have to deal with getting and using an ISP.
And kind of sad that it’d be so different for the AMD FW13. Would be sad to invest time to add such a feature only on the old branch / seems fan curve is already much smarter for the zephyr-based builds.

@DHowett do we know how far away the version of the EC firmware in the newest 12th gen firwmware release 3.08 is from the public hx20-hx30 branch? (4ea1c89, not in public github).

Are there any advancements visible for the newer boards in terms of doing EC firmware development without having an ISP setup? Like a 2nd or temporary firmware slot? So one could try a custom firmware, but when that fails to even boot fall back on official & working firmware that was never overwritten instead of having to open up the case and flash manually?.

And have I just missed it or is there really no way for the EC to communicate to the input modules? Such that the ability to control keyboard backlight from the EC to coordinate with brightness sensor, touchpad interaction, keyboard interaction etc. was actually lost with the addition of all that (rather gimmicky) input module modularity? Does not really look like FW planned for a 2nd communication channel between EC and input modules other than the simple sleep line.
Would also mean, that the EC could never coordinate keyboard backlight between multiple modules (numpad & main keyboard for example), and that’d have to be done with custom tools running on top of the main OS if ever. Unless one goes really crazy with that single sleep-wire or some other more hacky solution.

But, I really should complete my own stuff anyway, before even thinking of risking my notebook and delving deeper into this…

Wouldn’t you need a proximity sensor?

Assuming you mean the feature by which the keyboard enlightens when you approach your hand.

No, not that fancy. Using the existing brightness sensor, by which the screen brightness is also automatically regulated.
Just like my previous Dell XPS had: if its bright, keyboard backlight goes off. And comes on when warranted by it being dark. No manual turning on and off needed (but of course possible). Should really be combined with the timeout. Such that the backlight also goes off if you are watching a video etc. and do not interact with the device (keypresses, touchpad interaction). And any interaction should restore it to what it was previously.
Since all of that goes through the EC already on the FW13, it should be very possible to add this in. The brightness sensor is already regularly polled.

1 Like

Understood!
By the way, replacing “approaching the hand” by “touching the touchpad”, one can already make a small script for that on Linux.
Of course it’s even better in the firmware (frees CPU cycles).

To be fair, I actually do sign my binaries with a key stored in an HSM - my certificates just don’t roll up to a known CA. I’m looking into Azure Trusted Signing at least to ensure that I have a trusted CA, and we can work from there.

The major issue with the EV certificate isn’t the cost (though $500 to $700 per year for a hobby project is prohibitive) exactly, it’s the requirement that I am a business. I was unable to find a provider who would sell me an EV code signing certificate as an individual or even a DBA. Incorporating is cheap and easy in my state, but invariably they’d want a Dunn and Bradstreet number and D&B is effectively a protection racket for small businesses.

I’m still figuring out what to do here. An acquaintance of mine was willing to sign it so long as I implemented an allow-list for which host commands I’d pass through. I wasn’t interested in doing that. :slight_smile:


Oh, it’s most definitely not. :slight_smile:

Let me poke around a little bit. It shouldn’t be that far off. I know that for hx20, it’s the latest. I’m sure we can get hx30 pushed!

Every device using this embedded controller technically supports two firmware slots. On the lotus and azalea (the AMD devices), it works perfectly fine (ectool reboot_ec rw will take you to the RW firmware, ... ro will take you back.) On the hx- boards, it’s a little more complicated.

There’s a bug in the bootloader that prevents hx20 and hx30 from ever loading the RW firmware.

For hx20, this pull request fixes it. It’s broken down into commits, each explained.

Somebody would need to do similar for hx30 and test it out. They’re very closely-related, so it should be fairly easy.

The firmware I run on my 11th Gen is a combination of Framework’s upstream plus this fix for the RO image, and whatever wild stuff I’m working on for the RW image. Doing it this way carries all the risk of reflashing your firmware once, and then you’ll always have the mostly-untouched RO build to fall back on.

If RW crashes, it will automatically reboot to RO. It’s pretty slick.

I added a host command that lets me whack the bootloader in memory to test stuff like the RW fix. That works because it stays resident for the entire uptime of the EC.

Alas, I can’t say - it doesn’t look like it, but I don’t have a Laptop 16 in front of me to poke at.

hx20 has the ALS connected directly to the AP’s i2c bus, and the EC may be unable to read it. For hx30 and the newer boards, it should be possible! (EDITED to fix inaccuracy about where the ALS is connected.)

As for ISP, the JECDB connector is documented! :smiley:


As a quick note: this commit in my ectool fork adds PD controller version reporting (command fwpdversion). It’s only useful if you are worried that something happened during an upgrade.

I have not looked deep enough into Chromium-ECs bootloading. Is there an early way to switch those slots back? Like when the alternative image will not get to the console where you issue the reboot command (over UART if the AP does not boot).

Oh, nevermind. I should’ve kept reading. So the recovery back to RO is based only on unclean shutdown? Mhh. Ideally, the LFW would also have a way to manually override to the safe version. Like a specific key pressed during boot. Or use that UART that I bet is currently inactive to command it to override the slot (maybe only wait for UART under a specific key-press).

I only looked at hx30, because that is what I have.

Yeah. But soldering that connector in on that board is a big deterrent for me. I am so much more of a SW/FPGA guy. That sounds like I’d really only be comfortable to try, if I am ready to replace the entire board if need be or have already replaced it as my primary notebook (in which case it would be way less tempting to work on firmware additions that are not directly portable to newer devices).

Oh, well. One less reason to procrastinate my other work…

Good news: there is another mechanism to force a fallback to RO (disconnect main and RTC battery power, which will clear the battery-backed RAM where it stores the last boot system image) and UART is not inactive!

Ok, that sounds better. Especially because there is already a pull request that ported your work enabling ro/rw to hx30.

That post of yours I already read. With inactive I only meant during LFW. Where there is ifdef’ed code for also accepting a few commands from UART. But only reset, continue. Immediately before it tailjumps into one of the firmware slots. So I was only thinking in that direction. To have a non-intrusive way to force it back. But popping the RTC battery does not seem that much worse. Especially because it is the fallback for the fallback.

The new Trusted Signing service looks promising, but it seems to be only for businesses as well:

“Trusted Signing at this time can only onboard Legal Business Entities that have verifiable tax history of three or more years.”

Meanwhile, I found out what practices other people use to avoid signing their own drivers:

  • ec-probe.exe is using the old WinRing0x64.sys driver to access the EC memory. That driver was signed in 2008 and therefore still passes Secure Boot. If you Google it, it seems to have a bad reputation as it was often used in malware, so the new versions of the driver are functionally limited. That’s why people still use the old version.
    Note: ec-probe dump and FauxECTool give me a similar memory dump, but the address offset is a bit shifted (the dumps are only partially overlapping). I assume it cannot detect the correct EC address.
  • TPFanCtrl2 relies on another super-old driver called TVicPort; despite being from 2005, it still runs in Windows 11.

I briefly looked at your Windows driver source codes, but I don’t think I have the necessary experience and capacity to write any kind of a wrapper library that would expose the same API using an existing signed driver. I guess it’d be hypothetically possible to achieve, though. :man_shrugging:

M$ want control, in the name of security

Exactly

Is there something holding up that PR?

Do hx20 and hx30 correspond to different Framework models? I have the 16 but I have no idea if it’s hx20 or hx30 or something else.