I can imagine 2 would be fine as cooling is on both sides, I expect >2 might be a problem! I’ve had to put mine on hold for a few reasons, but now I can get expansion cards for a bit cheaper I’ll pick this back up again!
@Andy_Clemenko how’s it shaping up? I know there’s a lot of discussion above around cooling, and yours is pretty much worst case. I’m also curious about noise.
(I’m interested in building my own for home lab reasons–need a cluster test bed. Definitely thinking about including batteries, but I did read the above warning.)
The “BladeRunner” is great. There are ZERO heat issues. I was thinking about changing the orientation of the case. But since its working…
Do you have any designs for the case yet?
Blinking LEDs can be turned off in BIOS, just enable standalone operation.
Aside of that, have you considered connecting the nodes using Thunderbolt4 networking?
Yeah, nrp responded here:
Not yet, I’m learning CAD. My plan is to start with the mainboard enclosure that Framework posted to GitHub and extend it for battery, add notches for the ethernet cards, add some side fan vents, etc.
I haven’t heard of this! It would certainly be cool to get TB4 speeds instead of gigabit. But I’m only seeing it suggested for point-to-point (such as transferring files between two laptops), and I don’t think there’s thunderbolt routers. But having full power and ludicrous speed data on one little cable would be pretty slick.
all requires a dongle. thus the 2.5gb ones from framework
Thunderbolt theoretically has a point to point mode that looks to the system like Ethernet, not sure how well it actually works since I only have one laptop with thunderbolt but it is theoretically a thing in the thunderbolt spec.
If you just want to connect framework mainboards to each other you could make some kind of mesh out of tb p2p connections but it definitely would not be trivial.
but if you actually want to go to a wider network you do need a network card though, can be one of those 300+ buck tb 10gbit ones if you want I guess. Could also just go with a <100$ 5gbit usb one or a <30$ 2.5gbit one or a <10$ gbit one XD
You can directly connect two mainboards with a TB4 Type-C cable and it will work. No need for external devices.
For example, given 4 TB ports on each mainboard, 4 nodes can be connected into a full mesh.
I’ll have to test. But all my clusters also have a gateway router.
Anyone found a good multi-device power supply? The framework brick is pretty stackable, but it would be nice if there were even more compact options.
I did do a reduced version of the first-party printable case as my intro to Fusion. Simplified geometry, removed the wifi bump, added cutouts for for the ethernet module. I’ll be happy to share it, but it’s really not markedly different from what Framework published to GitHub. (Like, you can get the same functional differences with a dremel. Or a nice pair of flush cutters.)
Next version is going to add a battery area, improve air flow, use actual screws for assembly, maybe have a bracket for the power brick.
https://www.amazon.com/gp/product/B09PMDZWZ6/ worked for 3 devices. I used 2 of them to power a 6 node cluster : BladeRunner - Framework Laptop Cluster - k8s shoe box - YouTube
Brief update, got the physical proof-of-concept build done
I have thoughts about what I’m going to put on there (debian, figure out puppet, and then poke at container systems), but that feels like a long way off.
Currently playing with software. Hitting network weirdness.
You need to enable all the repos so you can get realtek-firmware
.
Hit Debian 11 - Ethernet Expansion Card Doesn't Work after Install
But I am having a lot of success playing with serverless puppet.
From a hardware perspective:
- Type C ports aren’t super robust to motion, esp with free cables. Using a battery or a rigid setup is recommended.
- I’m getting some weirdness around boot and my thunderbolt dock. It might be fixed in the later generations and/or newer firmware. Playing with replugging power/brick seems to work.
- Using a cable like https://www.amazon.com/dp/B0BJYW2FKG helps a bunch
I’m having such a hellish time with network adapters and Debian that I’m thinking of abandoning this project.
On Debian 12:
- RTL8156BG (the framework ethernet module): Debian can read the mac address, but that’s about it
- RTL8153A (an ethernet dongle i got for my work mac): Negotiates gigabit but doesn’t actually detect an “up” link
- RTL8153B (thunderbolt dock): Works great
I share my homelab with others, so I’m pretty married to Debian. My suggestion to others is if you’re going to attempt this, either don’t use Debian or don’t use Realtek.
Currently, I’m praying that Realtek’s Linux team fixes this.
I love this idea. I’m still thinking how to properly fit and cool 10 mainboards in 4U. Sounds like the perfect setup for a homelab.
For powering these I’m currently looking at https://www.bravour.com/en/10-ports-usb-c-65w-1u-rackmount-charging-hub.html . @Andy_Clemenko maybe this is also the thing you’re looking for?
My biggest struggle is still cooling. Would it be feasible to have a smaller rack (e.g. a 6U or 8U wall-mountable ) and have the exhaust at the top of the case? And intake from either the side or front. Not really the best option for a datacenter but perfect for a homelab use. Any thoughts?
Cooling is not a big issue. Have all the heat sinks vertical and vented out the top. The biggest hurdle in my mind is the power distribution. I would to figure out a server PSU to PD. 10 boards at 60w each. I wish there was an easy to adapt a 1000w server psu.
The other idea is a AC rail with 10 60w PD adapters.
How did you figure the power would work?
I’m planning to just use the power supply from Bravour that Iinked in my previous post.
It’s capable of delivering 10x 65w over USB PD and thus seems fully compatible with the framework mainboard.
Hi guys! Im working on a similar server concept with the exception that i only want one board and about 6 sata 3.5" hdd, my biggest issue so far has been finding a single power supply for both the board and the spinning rust to run off of, Id love to hear some advice on what could power it.