I’m waiting for my first board to arrive, but I’m holding up to see if prices will improve before buying the next cluster nodes.
I’m trying to collaborate with a proof of concept driver for RDMA over USB4 using the two back usb c ports that have 2x2 usb lanes, and would kindly ask if Framework can help me test it on their lab clusters.
The idea is to allow the driver to run RDMA low latency copies and emulate GPU Direct to save memory copies on the device.
My patches are in the form of creating robustness and validating topology with lane aggregation.
This driver would be then consumed by ROCm RCCL and rocSHMEM to allow tensor parallelism or future grid like architectures.
This is not an ask for sponsorship, free hardware or marketing commitments, the repo is already public and open. I’m asking if this can be tested if useful. Of course that I’m open to discussing it further.
Nice, I think that the latency is probably terrible on 5gbps USB3 but the ibverbs (Infiniband Verbs) don’t care about their transport layer and there’s a PoC of a driver using 2x2 channels for 40gbps over good USB4 cables here: thunderbolt-ibverbs: We have InfiniBand at home
Yes, I know, I have forked the work from there and been fleshing out some more. The lanes should allow near 10Gb, but topology, aggregation and size of packets/memory range is important to keep the BW high, latency is probably not possible to go lower than George original tests. I have been clear and sharing back my ideas to the original creator of course, just not polluting the repository. You can see my fork from there.
Meanwhile I’ve made a fork also from Donatto’s toolboxes to pull this project on RCCL. It builds fine on my side but would need help to validate on a cluster
Thank you, George has done all the hard work and initial batches, but this is promising for a “just works” solution as the hardware is quite capable. The idea is to have it open for the community to test, experiment and improve upon. Thank you and team for such a wonderful design and completeness of vision. It’s a joy to work on a device that is easy, stable and reliable.