The only product data that has been observed is a list of serial numbers for the components that the product shipped with (which the EC firmware neither reads nor writes) and a region containing the date of first boot (which the AP firmware writes directly to EC flash and is likewise not read by the EC firmware).
reflash subcommand (which is not the same as upstream ChromeOS’s ectool and its
flashwrite subcommand) preserves the VPD region.
Good catch with the WSON-8 chip.
Unfortunately, my probe arrived with a pin broken, so I had to hold a wire touching one of the pins, so my hand is tired, and I couldn’t verify the downloaded image against the flash chip again.
I have two questions, hoping people can help me with:
- why is the flash so big? it seems to be 32 MB of space available, of which the EC firmware is only 512 KB
- if I need to restore from an EC backup that I create using ECTool.efi, what offset should I write it to?
From some light spelunking, it looks like the flash chip’s storage at offset 0x1000 and length 0x80000 are identical to the ECTool.efi-created backup, so maybe that’s the answer to #2. I would much rather write just that portion in case my hands shake and I lose the connection.
Whoa! The flash chip for the EC firmware should only be 1MiB in size. The image from the ECTool.efi backup should match from offset
Where did you find information about the expected size of the flash chip? Is there a parts list or something for what I should find in my 12th gen laptop? If it should only be 1 MB, then it means I might not have found the right chip on the board… but it’s a strange coincidence to find an exact copy of the firmware in that chip. What is it for?
Also, the EC flash configuration on the Framework EC github page (sorry, can’t link. it got flagged as spam) says:
Currently the EC only runs from the ro region.
that 0x1000 offset matches where I found the matching portion from the ECTool.efi-created backup. Though the ending offsets of the regions don’t quite match.
I guess basically I need help figuring out where the flash chip is, haha.
OK, looking at the schematic, I see there are two SPI ROMs, a 1 MB one connected to the EC and a 32 MB one connected to CPU. Looks like I found the latter. I have a sneaking suspicion the EC and the 1 MB flash are under the fan or something. Really hoping they’re not on the bottom side of the mainboard… Does anyone know?
After hunting around on the top side of the mainboard a bunch, I finally decided to pull it out and look at the underside. There are actually three of the same model flash chip on the bottom: W25Q80DVIG. Looks like this is a 1 MB flash part, so that matches the schematic and earlier replies.
This chip is SOIC-8 but they’re all mounted almost flush with the board, so my SOIC-8 clip doesn’t fit. I’ll need to get a probe or something instead. There’s one quite close to the EC MCU, so that’s the one I’ll try first.
When I eventually write up something about this whole deal, I’ll include all the pictures I took, because there’s a real shortage of pictures of the bottom side of the mainboard.
Nope, another WSON-8
Hmm, the web page says SOIC-8 (and some other things, I guess), and my WSON-8 test clip doesn’t fit. Is it because it’s a WSON-8 6x5 instead of a WSON-8 6x8?
That’s just a list of the packages that the chip is available in. It can be bought is any one of those listed.
I have a mixed update. It took a few weeks for the WSON-8 6x5 test probe to arrive, but it came yesterday. I was able to use it to pull the contents of the flash closest to the EC on the bottom side of the mainboard, and the first 0x80000 bytes exactly match what ECTool pulled in its backup, so that looks good.
The other two flash chips on the bottom have what appears to be other firmware for some other components. Both have very different contents and none of them match the ECTool backup, so I’m just going to leave them alone. If someone wants, I can post the contents of those chips somewhere.
So, feeling confident I have an escape hatch, I reassembled my laptop, and… it wouldn’t charge or boot. It had been disconnected from battery for like 3 weeks. I was pretty careful handling the mainboard, so I’m not sure what the issue could have been.
I saw some suggestions that I should pop out and in the CMOS battery and try trickle charging it. Unfortunately, I cracked the CMOS receptacle, so Framework is luckily replacing the mainboard for me. When I get that replaced I guess I will get new flash backups and then finally try the flashing.
While I’m waiting for my replacement mainboard, I had a question. Is there any reason why touching the test probe and doing “read” and “verify” operations from the flash programmer could mess up the mainboard? I still don’t know why the first one wouldn’t charge and want to make sure it wasn’t because of my read and verify to create the backup. I didn’t do any erase or write operations.
I can’t think of a reason that would cause a lasting problem. However, if you are using an 11th gen laptop you’re probably seeing the fairly well-known RTC battery bug. Sometimes it requires trickle charging the entire machine (not just the RTC battery) from a 5V source rather than a PD source.
I’m really glad you were able to verify the flash! Sorry that you ended up getting held up by unexpected tech trouble
Mine is 12th gen, so I didn’t think that was the problem. I wasn’t able to trickle charge it with a non-PD charger (or I didn’t wait long enough before I took matters into my own hands and broke the part, oops). Thanks for the response. New board is on the way, so I’ll see how that goes soon.
Well, everything is good now. I received the new mainboard and it works. I was able to use ECTool to reflash the modified EC firmware… had a bug there, was able to restore from backup and fix the bug and reflash again, and now my laptop is working like I always wanted. I ended up not needing to use my backup, but I’m glad I had it just in case, and I’m pretty confident it would have worked.
At some point soon I will write a blog from this because I don’t go through this much angst without producing something out of it, haha. Thank you to all who helped with various information and advice.
I’ve written up the entire process, including the bug fix and some debugging. I link to pictures and flash chip dumps that I uploaded, too.
Thanks again to everyone who helped out on this thread. For @DHowett, I link to a proposed minor feature for ECTool. For the Framework team I list a few recommendations at the end.
This is amazing! Thanks for writing it up, and I’m glad we were all able to be part of the journey!
Ah, and if you submit a pull request for ECTool.efi I’d be happy to give it a look
Thank you for the effort of writing up this interesting journey!
Concerning the original behaviour that prompted your investigations: oddly, under linux with the 3.17 (11th gen) BIOS I am not seeing the keyboard behaviour you report. Instead I see:
- press “A” – key starts repeating after a short delay
- press “B” - a single “b” appears; starts repeating after (same) short delay
- release “A” – repeating stops, but after (same) short delay, repeating of “b” resumes
- release “B” – repeating of “b” stops (as one would expect)
So either the keyboard behaviour changed somewhere prior to 3.17 or linux handles the key press/release messages differently to produce repeating behaviour from it.
I guess I hope it’s linux; otherwise it’s rather tragic to go on a wild firmware goose chase only to have a firmware update void the work – although it does make for a better story if that were the case; and you (and the community) learned a lot along the way.
Huh, weird. I don’t see the EC firmware having any related updates recently, so I don’t really have an explanation there. It’s possible some of the “typematic” functionality is being handled in OS code, I suppose, so it could have linux-only behavior. I’m okay with how things turned out for me, though. I’m quite happy with my fixed laptop, it’s a good feeling to put a little “sweat equity” into it, and yanno, journey before destination, right?