Howdy,
has anyone tried making a ZFS mirror (RAID1 equivalent) on the laptop? I’d be interested in either M2 and expansion card and 2+ expansion cards mirrors (/ RAIDZ).
I’m mostly concerned about latency, since USB doesn’t have exactly stellar reputation in that department. So… if anyone here tried mirroring the root partition I’d love to hear how it works in practice and see some bench results 
Does the whole idea sound sensible without a second m2 slot?
Probably not a good idea, I’d say you’d probably either have it not work, or it will cripple the performance of the M.2 Drive. Also a PS, I don’t see why you’d need to mirror drives for a laptop. If it’s for loosing data then you should probably invest in ensuring everything gets properly backed up using a tool like ansible or manually backing up files.
It’ll definitely throttle down the write performance of the M.2 drive to that of the expansion card. I’m interested how in usable would the resulting storage be for a daily driver laptop (I’m aware that’s a quite vague workload description) due to USB latency. It’ll definitely be worse than 2xM.2, but maybe good enough.
As for the mirrors: my RPO for a single drive failure is somewhere around a single block. Given a single disk, total fault of the system is a certain event at some point unless you plan on swapping that single drive while it still works (which doesn’t give you certainty and generates waste). There is a few orders of magnitude smaller possibility of 2 drives failing before I replace the 1st faulty one and perform full resilvering. In that (unlikely) case I’m happy with RPO of 24h, which is how often I do my backups to external storage.
I can absolutely imagine someone having a different value for those RPO parameters, in which case a single drive + backup might suffice.
im not really a fan of mirroring SSDs.
SSDs don’t have as many failure modes as HDDs and the primary failure mode of SSDs is wearing out the flash cells. if you mirror SSDs then both will wear out at exactly the same rate and consequently will probably both die at the same time, defeating the object of the mirroring
Brace yourself, giant wall-of-text!
As I’ve mentioned elsewhere on the forum, I think it’d be theoretically possible to create a sort of m.2 2280 riser card that includes two m.2 2230 slots, both assigned 2 PCIe lanes each.
I have literally never had this happen. Conversely, I’ve had at least three SSDs just suddenly die, one of which I was able to revive by leaving it plugged into power for an hour, unplugging for an hour, and then plugging it back in. Admittedly though, all 3 SSDs were older models with capacities measured in mere double-digit gigabytes.
That being said, even just plain old power failure can brick an SSD - I have a family friend where that happened to a 480GB Crucial M500 SATA SSD after their desktop PC’s PSU failed (though, rather than being a true brick, it was permanently locked to a read-only state).
I similarly had the exact same “permanently locked to read-only” issue occur when trying to clone the data from the aforementioned family friend’s SSD when I was experimenting with powering two SSDs off of a USB power bank due to questionable weather at the time. I was almost certain it’d be fine even with both sharing a single USB port since the power bank can output up to 2 amps at 5 volts, but I made the horrible discovery that only one of the bank’s two USB ports can output at 2 amps and, sure enough, I connected them to the port that could only output 1 amp. Now luckily one of the two connected SSDs worked without issue to the point that I don’t even remember which SSD of mine it was, but the other SSD was ironically yet another 480GB Crucial M500 SATA SSD that, due to insufficient power, permanently locked itself to read-only.
(yet another reason for open firmware - I would love to be able to unlock the two aforementioned M500 SSDs that are locked to read-only and then run a proper test to see if they’re at least partially usable or not, but Crucial seems to have not made such a thing possible and AFAIK nobody has created any 3rd party “homebrew” firmware - even doing a secure erase doesn’t unlock them despite actually completing the actual erase process)
Lastly, I also had a psuedo-brick occur with an older 256GB Crucial M4 SATA SSD when I tried connecting it via a USB 3.0 SATA adapter that doesn’t have supplementary power to the front USB of an older LGA1156 motherboard, forgetting that most pre-USB 3.0 PCs only provide something like 500 milliamps to their front USB ports. Ironically, because it was an older model SSD that didn’t have the “lock to read-only” fail-safe, I was actually able to revive it by simply leaving it connected to power for an hour or so, unplugging it for another hour or so, and then plugging it back in.
One of the benefits of software RAID is that you can mix and match completely different drives in order to avoid this very issue, and is something you can do with mechanical hard drives as well.
And the good news is that, unlike mechanical hard drives, it’s so incredibly rare to have an SSD fail during a process where you’re only reading data (e.g. rebuilding a RAID mirror), so using two different model drives should basically guarantee you always having at least one functioning drive (assuming no catastrophic disaster has occurred such as the aforementioned PSU failure).
And technically, even if they’re the same model drive and you avoid any sort of “instant brick” failure scenario, if they were manufactured at substantially different times (e.g. a year or two apart) then it’s very likely that the NAND chips themselves have different yield characteristics - heck this could be the case even if they were manufactured close together simply due to the way the silicon lottery works (something that doesn’t apply for mechanical hard drives but definitely does for NAND flash memory).
@NM64 wow, kudos for the detailed reply, it’s like you’ve read my mind ;d
The riser idea is certainly interesting, but do you thing there is enough space to actually fit 2 nvme ssds and the riser itself inside the chassis?
I will admit that I have no idea how much Z-height there is to work with. At least on the bare motherboard used outside of the chassis (which is of course something the Framework laptop’s motherboard supports), on a desktop motherboard, or other device with enough Z-height to fit m.2 heatsinks, it shouldn’t be an issue.
That being said, at least the X and Y dimensions shouldn’t be an issue, and you can even save a bit of space by having the two 2230 SSDs facing back-to-back in opposite directions, allowing you to only need a single screw (or screw-less mechanism) placed directly in-between both SSDs to hold them both in place.
(of course you’d still need a second screw or screw-less mechanism to hold the 2280-sized rizer onto the motherboard itself)
EDIT: Or, as another fun idea, the screw holding the two 2230 SSDs in-place could be aligned with the standard m.2 2242 alignment so that all you’d really need is one, slightly longer screw that goes through the riser’s entire PCB, thereby it not only would hold the two 2230 m.2 cards on to the riser but would also hold the actual 2280 riser itself on to the actual motherboard!
…of course, the current Framework laptop motherboard only has a screw hole for 2280-length m.2 cards. 
It is a great idea, I already do it, and it is possible in the new frame.work laptops. I would like to confirm that both M.2s have full CPU capacity. Also good if you can get a non-QLC NVMe for this, and having a mirror instead of a single NVMe will increase ZFS’s performance. It is a shame that the ZFS devs aren’t using ZFS on their own machines; they are dog fooding. Bad engineers, bad! When they start doing this, we will have much better NVMe performance with ZFS and the stereotype of “this great filesystem for data integrity isn’t for end users cuz you guys have computers (SSDs) which are too fast.” For now you must just create the pool with the recommended parameters for your hardware and you’ll have OK performance.
Should work perfectly in the new frame.work 16 via 2x 2280s, eh?