Making use of FW16's storage capabilities

[Epsilon_Z]

Hello! I’ve been trying to consider how to best make use of the FW16 having at least two storage drives, or even more with expansion cards. But finding myself a bit overwhelmed by all the different ways I’ve found to handle data, I’ve decided to reach out here for further guidance. Here’s where I’m at:

Of course, the most straightforward way is to simply use all those storage drives as separate storage drives. But you could also stripe or mirror data across the internal drives with btrfs or zfs. But is striping worth the risk of losing all your data if a single drive fails? Is mirroring worth the loss of a whole drive’s capacity, especially if this is your main, or even only machine?

If a raid array is even worth it on this device, then with additional expansion cards, it should be possible to even go for raid 5, raid 6, or any of the different raid z setups, right? But if you did make a raid array that included expansion cards, would there be a performance hit, and how bad of one? Would it depend on the specific configuration?

And what if the drives are different sizes, such as the expansion cards possibly being smaller than the internal drives, or even the internal drives being different sizes? My understanding is that raid would no longer work. But what if, say, you had 2 TB internal drives and two 1 TB expansion cards? Could you use the two expansion cards as a single logical volume of the same size as the internal drives and then use that to make a raid array with the internal drives?

If expansion cards in a raid array would impact performance, but if they could also be grouped together within that array, could you put the expansion cards in raid 0 array nested within the larger main array to improve performance? Would performance still be worse than simply not having the expansion cards in the array? If a single expansion card failed, would it be possible to identify which one and only replace that one, or would you be forced to replace all the expansion cards used that were grouped together? If such a setup is even possible, would there even be any merit to having such a complex configuration?

What about using expansion cards to backup data? Is that a good idea, or does the fact that it’s still in the laptop make it useless?

I don’t know what all the pros and cons and considerations of all these possible setups are, or if there are other possible setups I’m not even aware of that are worth considering. All of these things are still very new to me, so I apologize if I’m getting basic concepts wrong, but I’m eager to learn! And so I would very much appreciate hearing other people’s thoughts on this and how you all intend to make use of your FW16’s storage!

[Dr_Kens]

1. Individual Drives vs. RAID (Striping/Mirroring):

• Using individual drives is straightforward but might not offer the performance or redundancy benefits.
• RAID 0 (striping) increases performance but comes with the risk of data loss if one drive fails.
• RAID 1 (mirroring) provides redundancy but at the cost of using half of the available storage.

2. RAID with Expansion Cards:

• Yes, it’s possible to create more advanced RAID setups like RAID 5, RAID 6, or RAID Z with expansion cards.
• Performance impact depends on factors such as the RAID level, the number of drives, and the specific configuration. Generally, RAID 5 or RAID Z1 may see a slight performance hit compared to RAID 0.
• Mixing drives of different sizes is possible, but it usually results in the total array size being limited by the smallest drive.

3. Nested RAID Arrays:

• It’s technically possible to create nested RAID arrays (like RAID 0 within a larger RAID array). This can be complex and might not offer substantial benefits, and it adds more points of failure.

4. Identifying and Replacing Failed Drives:

• In most RAID configurations, you can replace a failed drive with a new one. However, this process depends on the RAID level and the specific RAID controller or software used.
• RAID 5 or RAID Z2 allows for multiple drive failures without data loss.

5. Backup Considerations:

• RAID is not a substitute for backups. Even with redundancy, data loss can occur due to various reasons (e.g., accidental deletion, corruption).
• Backing up to external drives or cloud storage is a good practice for critical data.

6. Expansion Cards for Backup:

• Using expansion cards for backups within the laptop is a valid option, but it might not protect against certain types of data loss (e.g., laptop theft, hardware failure).

7. Complex Configurations:

• Complex setups can offer benefits in specific scenarios but might be overkill for general use.
• It’s crucial to weigh the complexity against the actual needs and potential risks.

In summary, the best approach depends on your priorities—whether it’s maximizing performance, ensuring data redundancy, or a balance of both. Additionally, having a solid backup strategy is crucial to safeguarding important data. It might be helpful to start with a RAID level that aligns with your needs and expand or adjust the setup based on your experience and requirements over time.

[Alan_Pearce]

@Epsilon_Z

But is striping worth the risk of losing all your data if a single drive fails? Is mirroring worth the loss of a whole drive’s capacity, especially if this is your main, or even only machine?

Mirroring is only worth it if you have data sets that you want extremely rapid access to, or require higher reliability than a single drive can give you. BUT often people have a problem because both drives are from the same manufacturer and same batch and tend to fail at the same time, which doesn’t add to the reliability. You can only do mirroring with drives of equal logical capacity.

Striping is only worth it if you can fit a lot of drives so that the capacity loss through having one drive effectively as an error check drive (i.e. using 5 drives you get 4 drives worth of storage capacity). But again each logical drive needs to be the same capacity.

Using USB attach drives as part of an array is potentially fraught with problems. What happens if it gets disconnected, or the USB port has a problem? Such drives also tend to be significantly slower than internal drives, and as such will not help your array speed. Also the 1TB drive that FW are offering is reported to get rather hot, even when not being actively accessed, which sounds strange to me.

Quite frankly, how much drive capacity are you looking at? an FW16 with two drives will currently max out at 10TB (if you can find an 8TB 2280 drive which seem to have gone as scarce as hens teeth). The 2230 drive will max out at 2TB, and is slower (and more expensive) than some of the 2280 drives that will fit the other slot. But even if you use a 4TB 2280 drive that will give you 6TB of storage - and try finding a portable way of backing that up. I think you start looking at USB attach spinning rust drives if you want it all in one unit, or you get an nvme USB attach case and put another 2280 drive in that.

In short, any form of raid is not worth the effort unless you are looking for absolute data security, and then you are better off getting some large USB sticks or drives to use for incremental backup cycles.

Oh, and don’t forget the time it will take to do the backups, they are not instantaneous … !!!

[Loren_L]

Your questions basically boil down to a couple of recommendations.

If you’re looking to play around, and just experiment with how things behave, go at it. It’s a ball.

I’ve played with mirroring and striping. You’ll find that your limits will be dictated by whichever devices have the highest latency and/or lowest bandwidth. For most consumer products, like a laptop, you’ll generally only get into RAID 1 if your data is absolutely critical, or RAID 0 if you’re trying to get every drop of speed you can out of the bus. In either scenario, you’ll want drives that match either identically, or as close as you can get. Anything more complicated on a platform like this is pretty dodgy, and you’re better off just leveraging individual volumes for daily work and play.

If you really want to experiment with storage tech, and leverage it for real use, get a multi-bay home NAS, or a desktop with lots of drive bays. They will give you a better taste for how the options play in different scenarios.

By all means though, play all you can. Just make sure you have extra copies of anything important.

[txr13]

Like some have already said, if you’re just experimenting (and don’t care if you lose everything), go for it!

If you’re looking for practical advice for implementation, I would suggest this: don’t mix the internal drives with drives connected via expansion cards. Drives connected via the expansion bay are probably fine to mix with the internals.

I personally use mirroring on my desktop setups, but not on a laptop. This really comes down to personal preferences on risk tolerance and budget, for the most part.

Some of the details will also depend on the specifics of what you do. The options in a RAID-based system will differ based on your controller, and will also be different from the options you would get in a ZFS-based system, for example.

On my FW16, I will have one internal drive (running Linux) and one expansion card drive (running Windows), both encrypted and therefore unreadable to each other.

[Kellic]

It hasn’t been stated here yet but also keep in mind that mirroring doesn’t guarantee data security. Once upon a time I worked as a Dell field rep and replaced a drive that was in a mirror. The storage on that drive was corrupt because of a failing drive, that copied over to the other drive and BOOM…his entire dataset was hosed. Honestly solid state storage is reliable enough now that you don’t need to bother with mirroring, and fast enough you really don’t need to bother with striping either…just make sure you have a solid backup. Heck maybe even backup to an expansion storage you have plugged into the laptop’s port? Easy enough to setup in *nix or Windows.