Yeah, I agree, dedicated NICs with RDMA will be the way to go for multi node inference. IMHO this is a major design flaw in AMDs Ryzen AI Halo box. Nvidia ships it’s Spark with a ConnectX-7 for a reason.
That makes sense. So I guess we’re back to the question of whether the back of the Desktop chassis could include a PCIe expansion slot opening, along with the earlier discussion about using an open-ended PCIe slot. (Although I already use PCIE-oculink)
Well, the new 192GB desktop that has been teased has an open ended slot. It will be interesting to see if the case has an appropriate opening.