That still sounds rather unbelievable given where we are. Reminder:
- You listed an issue as fixed, that I reported as not fixed. Yet there was never any response or reaction to that. You have since released that version (2nd ReTimer update fails when using the 3.08x Windows updater is the issue) without further notes to users on how to handle this. Even if my setup was known to be the cause of that and so rare that barely any other customer would be affected, I should still be told what it is about my setup that caused that issue. And I believe I was not even the only one who hit that issue.
- You were notified of a bug in the 3.08 Windows installer. Reacted quickly with the fixed 3.08b Windows installer in January. Yet when trying to release the updater in response to bad press, you managed to release the old version without additional notes. It took 2 people posting screenshots of the updater failing on reddit for a community manager to take note and switch out the link. Even though multiple people in this thread posted about it long before. And I notified said community manager of said problem 2 days before the wider public started to take note. If you actually read through the thread and make notes of issues that are reported, I cannot imagine making that mistake for as long as you did.
- to this day, the release notes for the released firmware include trivial math mistakes like calling 2 different software version of the GOP firmware “same” (21.0.1061 vs 21.0.1046). This was also pointed out now months ago (apart from comparing between 3.06 and 3.08, even though 3.06 was never a release and is not otherwise known for customers only looking at the knowledgebase article as they will be on 3.04/3.05).
So I find it hard to believe that technical people read over the feedback here. Or if they do, they seem to not do anything with that feedback or it seems to be ignored somewhere.
Now, you clearly tried not to respond to my actual concern:
You said
It was announced as Alpha, I do not expect a great many of tests.
The point I raised, was that the 3.08b EFI updater as released CANNOT EVER update the right ReTimer.
Unless I am very wrong about how UEFI Capsule updates work (and I don’t think I am, otherwise I would not be writing this. And if I somehow turn out to be wrong I will change my opinion quickly), this leaves 2 options:
- Your testing procedures did not verify that the ReTimer was correctly updated / did not notice that the right side ports do not function (as I understand it USB2 and charging might still work in this case, but nothing else).
- A configuration where you tested the update from the prior release (i.e. 3.04/3.05 factory versions) was not among those configurations that were tested. In this case I would expect somebody to take note of not testing an update to work in the situation that I imagine is the ultimate goal of producing that update and not mentioning it somewhere. For example a “we have only tested on boards with 3.06 and 3.08, but we think it should also work with the officially released versions” would have sidestepped this whole issue.
Now, you clearly did also not engineer that installer only for beta participants. Because customers that ran the 3.08a EFI installer ended up in a state where everything was updated, but the ME firmware. As was reported many times in this thread before and after you pulled that installer. And this combination (everything but ME up-to-date) is not detected by the current 3.08b EFI installer. It instead claims everything is updated, even though ME is still outdated. So I can only imagine your goal in developing the new installer was only the official update path from released firmware version to released firmware version. Which is also the one that does not work. And if I am correct can never have worked.
This also leads to another concern that you seem to not care enough to give people enough information to get into those well-known states that you are designing and testing for (or not as it looks). Something that I also would have considered handled, if you made any public statement after reports of the ME update missing.
Coincidentally, this issue also means, the missing ME update was also not caught by your tests. Sadly, H2OFFT documentation is not public, so I cannot tell myself whether it should update ME firmware like Kieran implied. I only know that it has a commandline flag to update ME (that you have not been using). So I am curious: does it have a very unreliable feature such that it did not update ME for multiple people but did so reliably in your testing? And implicitly without the extra update-ME flag? Then it would make perfect sense why that problem was not caught by internal testing. Or is that like the ReTimer updates now, where you did not have a test that either verified ME updated correctly (as you will not notice unless you check the version) or not a test with older ME firmware that needed an update?
You have also received reports of the hardcoded filesystem path not working for everybody. And I even tested and concluded that this is most likely caused by whether the USB stick the installer is on is USB2 or USB3. The amount of other FAT32 partitions on the internal NVMe (that come before the EFI system partition) also affects it. So again:
- if technical people read this thread, a great solution would be to add a note of that. So that volunteers know when their setup definitely won’t work for the updater.
- If that is instead an area, where you are not even sure if the rules I found are complete and correct, then you should have asked people for whom fs0 is not the EFI system partition to report their file-system mappings and what USB stick they were using. So you have a chance of fixing it.
- If you in fact are aware of the complete rules for that, then it is not acceptable to still not notify volunteers about that known limitation.
In this case it also worries me, that the developer that hardcoded it in the same script that also searches all volumes for the USB stick itself, did not realize that hardcoding “fs0” was either a temporary stop-gap solution (that needs to be documented with a comment and noted when publishing the installer) or that the developer did not think that hardcoding something like that could ever be a problem. And nobody else looked over that small script and noticed.
The other issues with the alpha updater seem reasonable to me for an alpha-quality update (show success message even though it failed for example seems a much more niche case than the basic update from officially supported state to official supported state not working).
And to round out with a little more nuance:
- I am not directing this at specific people. I have no idea who does what and is responsible for what exactly. This is directed at the company as a whole.
- I personally do care a lot less about the speed. Yes the time so far has been a criticism. But I find Frameworks explanation for that plausible, now that we have finally gotten that: mistake in not planning for those resources and it took very long to compensate.
- I want a notebook that is modular, upgradable and repairable. I want to have confidence in that the product either has no issues or that Framework wants to and will fix issues it has with either hardware or software updates. This requires trust that Framework can deliver an update when it needs to. And will tell me, when a hardware upgrade fixes sth.
- What I care most about right now: It looks like Framework is not on a path that leads to reliably working updates any time soon. And you refusing to answer questions regarding that makes me only assume worse. Because the last time Framework was so very vague about things, the reality was the worst possible interpretation where you did not lie: you said you “will deliver an update”. And it took far over a year, dragging Framework by their feet with bad press and mistakes in even selecting the release that was produced months prior in order to actually do it. So if you make it actively look like you don’t see an issue, won’t change anything and I am also not wrong, there would be a very low chance of Framework ever having an installer that works for all the situations it needs to. And it looks like Framework wants to obfuscate that fact by not acknowledging it.
- These are issues of process that are probably affecting all other devices as well