How to prepare for reflashing firmware

knutaf · May 12, 2023, 7:46am

I’m rebuilding my firmware to try to fix a keyboard issue I’ve been having. I followed DHowett’s guide for building. I’m using Windows Subsystem for Linux. It seems to have built a correctly-sized binary. I followed the warning to use GCC 10. I’ve got ectool booting properly and showing version as well, but I chickened out before reflashing because I really don’t want to brick my laptop.

Before doing the reflash, what tools should I get or other preparation can I do to ensure I’ll be able to revive my laptop if something goes wrong?

For background, I have dabbled with a Raspberry Pi Pico and can solder stuff badly, but don’t have any real electronics experience.

In case anyone is curious what the issue is: on any computer I’ve used in my whole life except this Framework laptop, the keyboard repeat behavior is different.

press and hold A
press and hold B, while also still holding down A
release A
expected: B keeps repeating
actual: no keystrokes repeat

I found a place to patch it in the keyboard code in the firmware, so that’s what I want to try now.

DHowett · May 20, 2023, 1:05pm

Hey there! Sorry I let this thread sit for a while.

Honestly, I can’t recommend any specific recovery tools that would help if you really do brick the thing (i.e. get it into a state where the EC isn’t booting.) I believe you would be able to debug it via the JECDB header (0.5mm FPC (?); pinout described in Framework’s upstream EC repository), but I haven’t tried this myself.

I chose to jump in with both feet and absolutely no recovery plan . It is probably not for everyone.

Worst case, I think that you should be able to attach to the SPI flash on the backside of the mainboard with a chip clip (for example, this Pomona SOIC-8 clip) and hold the EC in reset while you reflash it over SPI. I should probably try this out before officially recommending it to people.

Funny enough, that key repeat behavior is exactly one of the reasons that I got into doing my own EC mods! I thought I was a crazy person for noticing it!

knutaf · May 21, 2023, 6:58am

Thank you so much for the reply. Directly interfacing with the SPI flash chip sounds a bit worrisome to me. I’ve never done that before. I see the JECDB seems to support SWD, so I’d like to experiment with connecting my Raspberry Pi as a Picoprobe.

If I can get that connected, do you think it would be possible to fix a broken EC by loading in a pristine backup into memory using a series of memory write commands from the debugger? That would depend on being able to get the debugger connected at a point where I can overwrite the firmware image in memory before it tries to execute it.

Or I if flash is mapped as a range of memory addresses, would I be able to use memory edit commands to write the pristine backup directly to flash?

knutaf · May 21, 2023, 7:54am

Hmm, just opened up my case and wow, I’m going to have to get some equipment if I want to microsolder this fpc connector, huh? No way I can do this with my soldering iron.

MJ1 · May 21, 2023, 9:18pm

A chip clip like DHowett mentioned is what people normally use for recovering chips from bad flashes. They aren’t hard to use, presuming there isn’t anything physically blocking you from attaching the clip. I’ve used an ultra-cheap aliexpress SOIC-8 clip a few times.

knutaf · May 21, 2023, 11:16pm

OK, thank you for that guidance. I found a bit more about the procedure from pages about unbricking a Chromebook. I assume since the Framework EC is a fork of the Chrome OS EC, this ought to be a similar procedure. It looks like that flash chip programmer can detect the chip type so I’ll know I’m about to make a mistake. I’ve ordered the programmer parts and now need to find the chip on the mainboard.

MJ1 · May 22, 2023, 12:14am

You mentioned having a Raspberry Pi, it seems you can use that too. Just need the clip and maybe a 1.8v level shifter. I don’t know if the Framework chip is 1.8v or not.
Ran across this page on using a RasPi Recover Bricked BIOS using FlashRom on a Raspberry Pi | Rototron

MJ1 · May 22, 2023, 12:29am

̶O̶h̶ ̶n̶o̶,̶ ̶i̶t̶ ̶m̶i̶g̶h̶t̶ ̶n̶o̶t̶ ̶b̶e̶ ̶a̶ ̶S̶O̶I̶C̶-̶8̶.̶ ̶@̶k̶n̶u̶t̶a̶f̶ ̶
̶h̶t̶t̶p̶s̶:̶/̶/̶g̶i̶t̶h̶u̶b̶.̶c̶o̶m̶/̶F̶r̶a̶m̶e̶w̶o̶r̶k̶C̶o̶m̶p̶u̶t̶e̶r̶/̶E̶m̶b̶e̶d̶d̶e̶d̶C̶o̶n̶t̶r̶o̶l̶l̶e̶r̶#̶e̶c̶-̶b̶a̶c̶k̶g̶r̶o̶u̶n̶d̶-̶i̶n̶f̶o̶r̶m̶a̶t̶i̶o̶n̶
̶
̶>̶ ̶T̶h̶e̶ ̶E̶C̶ ̶i̶s̶ ̶a̶ ̶M̶E̶C̶1̶5̶2̶1̶H̶-̶B̶0̶-̶I̶-̶S̶Z̶ ̶W̶F̶B̶G̶A̶1̶4̶4̶ ̶w̶h̶i̶c̶h̶ ̶h̶a̶s̶ ̶2̶5̶6̶k̶B̶ ̶o̶f̶ ̶R̶A̶M̶.̶
̶
̶T̶h̶a̶t̶’̶s̶ ̶B̶G̶A̶ ̶/̶ ̶b̶a̶l̶l̶ ̶g̶r̶i̶d̶ ̶a̶r̶r̶a̶y̶.̶
̶[̶m̶o̶u̶s̶e̶r̶.̶c̶o̶m̶/̶c̶/̶?̶q̶=̶M̶E̶C̶1̶5̶2̶1̶H̶-̶B̶0̶-̶I̶-̶S̶Z̶]̶(̶h̶t̶t̶p̶s̶:̶/̶/̶m̶o̶u̶s̶e̶r̶.̶c̶o̶m̶/̶c̶/̶?̶q̶=̶M̶E̶C̶1̶5̶2̶1̶H̶-̶B̶0̶-̶I̶-̶S̶Z̶)̶
̶
̶@̶A̶n̶y̶ ̶F̶r̶a̶m̶e̶w̶o̶r̶k̶ ̶s̶t̶a̶f̶f̶ ̶w̶a̶n̶d̶e̶r̶i̶n̶g̶ ̶b̶y̶,̶ ̶P̶l̶e̶a̶s̶e̶ ̶t̶e̶l̶l̶ ̶u̶s̶ ̶t̶h̶i̶s̶ ̶i̶s̶ ̶n̶o̶t̶ ̶c̶o̶r̶r̶e̶c̶t̶.̶ ̶G̶i̶v̶e̶n̶ ̶t̶h̶e̶ ̶r̶e̶p̶a̶i̶r̶a̶b̶i̶l̶i̶t̶y̶ ̶g̶o̶a̶l̶ ̶o̶f̶ ̶F̶r̶a̶m̶e̶w̶o̶r̶k̶,̶ ̶i̶t̶ ̶r̶e̶a̶l̶l̶y̶ ̶s̶h̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶e̶i̶t̶h̶e̶r̶ ̶a̶n̶ ̶S̶O̶I̶C̶-̶8̶ ̶o̶r̶ ̶t̶h̶e̶ ̶E̶C̶ ̶d̶e̶b̶u̶g̶ ̶c̶o̶n̶n̶e̶c̶t̶o̶r̶ ̶s̶h̶o̶u̶l̶d̶ ̶b̶e̶ ̶p̶o̶p̶u̶l̶a̶t̶e̶d̶ ̶o̶n̶ ̶p̶r̶o̶d̶u̶c̶t̶i̶o̶n̶ ̶b̶o̶a̶r̶d̶s̶.̶ ̶P̶l̶e̶a̶s̶e̶ ̶p̶o̶p̶u̶l̶a̶t̶e̶ ̶t̶h̶e̶ ̶d̶e̶b̶u̶g̶ ̶c̶o̶n̶n̶e̶c̶t̶o̶r̶ ̶o̶n̶ ̶f̶u̶t̶u̶r̶e̶ ̶F̶r̶a̶m̶e̶w̶o̶r̶k̶-̶1̶3̶ ̶b̶o̶a̶r̶d̶s̶ ̶a̶n̶d̶ ̶t̶h̶e̶ ̶F̶r̶a̶m̶e̶w̶o̶r̶k̶-̶1̶6̶!̶

Damn it, wait. That’s not the flash, but the controller itself. Major reading fail. I’d like to blame being sick, but who knows.

knutaf · May 22, 2023, 12:46am

Well, for what it’s worth I found this, which is at least a bit similar to the link I found before. Is the ec flash chip separate from the ec mcu?

Looks like a winbond 25R256JVEN.

MJ1 · May 22, 2023, 12:49am

You are correct. Major reading fail on my part!

Though, it’s still a problem if that’s the ec flash. That’s not SOIC-8

Josh_Cook · May 22, 2023, 12:50am

Looks like a WSON-8

knutaf · May 22, 2023, 12:51am

Thanks, I’ll check out the RPi method too. One thing I’m curious about is the mention of “VPD” (vital product data). When I extract a pristine copy of my firmware, theoretically it will have this included. When I flash a new firmware, is it going to have blank data for my serial number, MAC addresses, and so on? Do I need to somehow merge the pristine copy’s VPD with the new firmware? Or when I flash the firmware image will it overwrite only the fixed part and leave my per-device information alone? I guess that’s a question for @DHowett about ectool.

MJ1 · May 22, 2023, 1:09am

There are pogo pin jigs for WSON-8. Such as this or this.

DHowett · May 22, 2023, 5:21am

The only product data that has been observed is a list of serial numbers for the components that the product shipped with (which the EC firmware neither reads nor writes) and a region containing the date of first boot (which the AP firmware writes directly to EC flash and is likewise not read by the EC firmware).

ECTool.efi’s reflash subcommand (which is not the same as upstream ChromeOS’s ectool and its flashwrite subcommand) preserves the VPD region.

Good catch with the WSON-8 chip.

knutaf · May 31, 2023, 7:33am

Well, my WSON-8 probe arrived today, and I was able to use the CH341a programmer on a Windows PC with AsProgrammer and the right CH341a drivers to read the contents of the chip.

Unfortunately, my probe arrived with a pin broken, so I had to hold a wire touching one of the pins, so my hand is tired, and I couldn’t verify the downloaded image against the flash chip again.

I have two questions, hoping people can help me with:

why is the flash so big? it seems to be 32 MB of space available, of which the EC firmware is only 512 KB
if I need to restore from an EC backup that I create using ECTool.efi, what offset should I write it to?

From some light spelunking, it looks like the flash chip’s storage at offset 0x1000 and length 0x80000 are identical to the ECTool.efi-created backup, so maybe that’s the answer to #2. I would much rather write just that portion in case my hands shake and I lose the connection.

DHowett · May 31, 2023, 2:38pm

Whoa! The flash chip for the EC firmware should only be 1MiB in size. The image from the ECTool.efi backup should match from offset 0x0 to 0x80000.

knutaf · May 31, 2023, 3:57pm

Where did you find information about the expected size of the flash chip? Is there a parts list or something for what I should find in my 12th gen laptop? If it should only be 1 MB, then it means I might not have found the right chip on the board… but it’s a strange coincidence to find an exact copy of the firmware in that chip. What is it for?

Also, the EC flash configuration on the Framework EC github page (sorry, can’t link. it got flagged as spam) says:

Currently the EC only runs from the ro region.

00000000:00000fff bootsector
00001000:00039fff lfwro
00040000:00078fff rw

that 0x1000 offset matches where I found the matching portion from the ECTool.efi-created backup. Though the ending offsets of the regions don’t quite match.

I guess basically I need help figuring out where the flash chip is, haha.

knutaf · May 31, 2023, 4:14pm

OK, looking at the schematic, I see there are two SPI ROMs, a 1 MB one connected to the EC and a 32 MB one connected to CPU. Looks like I found the latter. I have a sneaking suspicion the EC and the 1 MB flash are under the fan or something. Really hoping they’re not on the bottom side of the mainboard… Does anyone know?

knutaf · June 1, 2023, 7:34am

After hunting around on the top side of the mainboard a bunch, I finally decided to pull it out and look at the underside. There are actually three of the same model flash chip on the bottom: W25Q80DVIG. Looks like this is a 1 MB flash part, so that matches the schematic and earlier replies.

This chip is SOIC-8 but they’re all mounted almost flush with the board, so my SOIC-8 clip doesn’t fit. I’ll need to get a probe or something instead. There’s one quite close to the EC MCU, so that’s the one I’ll try first.

When I eventually write up something about this whole deal, I’ll include all the pictures I took, because there’s a real shortage of pictures of the bottom side of the mainboard.

Josh_Cook · June 1, 2023, 7:45am

Nope, another WSON-8