IOMMU groups for Thunderbolt 4

[pyro]

One use case I will be testing, that I’m not sure if its been tested yet is running a VM on the Framework laptop then passing my egpu dock through to the VM. I can see a potential issue arising where maybe it will not be possible to separate the separate thunderbolt ports into different IOMMU groups and only passing the port for the egpu through to the vm maybe impossible.

I’ll let you know when I get mine and test, but just wanted to bring it up here to see if anyone else has tested this use case.

[Kieran_Levin]

We cannot officially comment on thunderbolt support yet.
With that said,
The laptop has the 4 physical ports split into two logical PCIE devices. With the ports on the left side going to one controller and the ports on the right side going to the second controller. So I think if you were ok with 2 ports on one side going to your VM, I think you could use the ports on the other side with your host OS.

I have not validated this, so no guarantees on this actually working or being correct. But if you get to test this out, it would be great to post your results.

[pyro]

@Kieran_Levin

The laptop has the 4 physical ports split into two logical PCIE devices. With the ports on the left side going to one controller and the ports on the right side going to the second controller. So I think if you were ok with 2 ports on one side going to your VM, I think you could use the ports on the other side with your host OS.

Can I just say that you guys being willing to make comments like this of the technical nature that tells us exactly what we’re asking, and explains it is incredible! Fore real kudos to you guys for #1 doing something that really needed to be done in creating a user serviceable laptop, and #2 for your amazing community engagement, keep up the great work!

[joevt]

@Kieran_Levin

The laptop has the 4 physical ports split into two logical PCIE devices. With the ports on the left side going to one controller and the ports on the right side going to the second controller. So I think if you were ok with 2 ports on one side going to your VM, I think you could use the ports on the other side with your host OS.

I think there is only one USB 3.1 gen 2 controller shared by all 4 ports. So maybe you could split the Thunderbolt PCIe root ports and USB4 controllers between guest and host OSs but USB 3.x functionality would be weird (if it’s not USB from a Thunderbolt device using tunnelled PCIe). I’m not sure since I don’t have any experience with Tiger Lake or with VMs that can take over a PCIe device. I know VMs can take over specific USB devices without taking over the PCIe controller so maybe there’s a work around if there’s a problem.

4 thunderbolt ports on the model? 40GBps on each port?

Framework uses Tiger Lake CPU which has integrated USB4/Thunderbolt 4 controller so it wouldn’t need a JHL8540 Maple Ridge discrete Thunderbolt controller. https://community.frame.work/t/thunderbolt-4/255/20 Maybe it uses JHL8040R Retimer? Anyway, with TigerLake Thunderbolt 4: There’s one USB controller for the USB 3.1 gen 2 functionality of all four ports. Probably this also gets used for USB tunnelling functionality of the USB4 controller when a USB4 dock/hub is connected. There’s two (I …

[pyro]

@joevt

VMs can take over specific USB devices without taking over the PCIe controller so maybe there’s a work around if there’s a problem.

Yeah that how I usually do peripherals for my vms, passing a USB controller in theory should be lower latency, but I’ve not noticed any latency issues just passing through individual devices so I just do that to make it easier.

[joevt]

For your eGPU use case, you won’t be dealing with USB devices because eGPUs use Alpine Ridge (or Titan Ridge) so no USB tunnelling will be involved. However, you still have to worry about USB devices connected to the other Thunderbolt ports (external mouse and keyboard? - you could try moving those to the eGPU’s USB ports).

Maybe it would have been useful to have at least one non-Thunderbolt USB port in the Framework Laptop (Tiger Lake has another USB XHCI which is separate from the USB XHCI used for Thunderbolt). I guess we’ll see what you discover with your VM testing.

[pyro]

Correct egpu doesn’t care about USB, but when you’re using a physical gpu instead of spice you still need a way to interact with the computer so you need to pass through thinks like a keyboard and mouse. I pass through a USB keyboard and mouse then use synergy to connect it back to my Linux host so the vm and the host share a keyboard and mouse, but when I’m playing games I can stop the synergy server and allow my game to have exclusive access to the mouse.