USB4/Thunderbolt vs Oculink vs something better for eGPUs

Hi,

USB4/Thunderbolt encapsulates the PCIe messages in USB packets. So this adds latency and overhead. Thus making USB4 docks slower than Oculink 4i docks.

When doing Display Port Alt mode, the USB packets are replaced by raw display port protocol over the same wires that send USB packets. This adds no overhead.

So, my common sense question is, why is there not a PCIe Alt mode to send raw PCIe protocol messages over the same wires that send USB packets. It would add no overhead, so why was it not done. The USB4 ports have some extra pins for USB2, so any of the plug/unplug pin up/down logic could have been passed over USB2, while keeping the high speed USB4 pins to be used for PCIe instead.

The only fairly obvious problem is lanes and bandwidth.
PCIe gen 4 is 16 Gb/s per lane.
USB4 is 10 or 20 Gb/s per lane.
USB4 is only 2 lanes.
So, USB4 would be more like Oculink 2i instead of Oculink 4i.
But, at least the latency would be better and cheaper with PCIe alt mode (if it existed)

Probably because of the whole hotplug thing and because using the wires would only give you enough for 2 lanes of pcie (separate rx/tx pairs) vs the slightly less than 4 lanes you can get with usb4 (Asuming you do the clock signal over some of the auxiliary wires, otherwise even less than 2 lanes).

There isn’t that much encapsulation (especially in the case of USB4) because thunderbolt is already quite similar to pcie. Pretty sure with newer usb4 chipsets that can do 256byte transactions instead of the old tb3 limit of 128 where it had to split each pcie message into 2 tb messages the latency and overhead reduced significantly.

Since the usb3 and usb2 controllers are pretty much entirely independent things that is probably less trivial than it sounds. You’d probably have to use cc for any negotiations as the side band pins would already be used for the clock signal, that would of course need custom cables.

Over all this would be extremely not user friendly compared to the relatively plug and play nature of usb4/tb. It may be cheaper (then again probably not event hat much since you need some very specially made cables and stuff) and you basically get a worst of both worlds, less performance than occulink and less convenience than usb4.

The holy grail would be some form of raw external pcie connector that supports hotplug (and maybe some usb and power). Something like wide usb-c with like 20 pairs, 8x pcie + 4 pairs for for usb or display.

1 Like

Another idea.
Use fibre cabling instead of copper cabling.
I saw this video and thought that maybe the prices are coming down enough to do some sort of optical setup.

PCIe gen 3 is 8 GT/s per lane.
PCe gen 4 is 16 GT/s per lane.

A fiber transceiver, about USD 100, can do 4 x 25 Gbps, so the equivalent of Oculink 4i.
Two fibre transceivers would do 8i.
Fibre cable is about USD 10.

That is still about USB 220 for the PCB each end and the Transceivers and Cable. So, still more expensive than the OcuLink cable, but it would remove all the signal quality issues and allow longer cables and one could probably make it pcie hotplug.

One would need to disable Spread Spectrum Clocking (SSC) as it is not needed for optical links.

What about PCIe gen 5?
PCIe gen 5 is 32 GT/s. That would need 4 x 50Gbps per fiber transceiver, that are about USB 600 each. So probably a little pricey.

Fiber Transceivers are thin enough to fit through the FW16 expansion bay opening at the back of the laptop.

Another interesting idea might be using PAM4, currently for PCIe gen 6, but maybe introduce it to PCIe gen 4 and gen 5 when going over copper OCUlink. It might reduce the amount of errors over the link. The use of PAM4 effectively halves the needed clock rate while maintaining the data rate while also adding in some extra error correction.

For fiber it’s probably better to go wider than faster as you can just use more separate wavelengths over the same fiber.

Over all I don’t think the physical transport is what hinders adoption but the lack of standardization and hotplug, both of which tb/usb4 “solves” which is why we got that and very little occulink/mcio coming from oems.

Maybe that chinese wide usb-c standard doesn’t turn out to be vaporware and gives us a higher bandwidth somewhat standard connector but I am not holding my breath.

Regarding hot plug. The question then becomes, what makes some connectors hotplugable, and some connectors not hotplugable?
I think it has two aspects:

  1. able to plug / unplug without causing shorts.
  2. way to detect the plug in the process of being plug / unplug so that it is electrically disconnected before the physical connections are fully seated.

This is normally achieved by having one pin shorter than the rest, so that it only touches when we are sure all the other pins are making a good connection and also disconnects before all the other pins, so the electronics can detect that an power off all the other pins before the user actually completes the cable pull.
All these electrical short, disconnect problems disappear with optic fibre links, because there is no electricity involved at the connector.
So, hot plug fibre links are very easy to do. You have one physical sensor to detect cable insert. Only power on laser when the cable is inserted. Job done.
That is only to protect peoples eyes from the laser, and is present on all class 1 optical network links.

Intent for the most part, pcie hotplug is a thing but very rarely implemented correctly, at least on the consumer side. Since thunderbolt madates hotplug it is pretty consistently implemented there.

So are copper ones, hell even the regular old pcie connector kinda can hotplug electrically but it is almost never properly implemented.