VFIO GPU Passthrough working on Framework - GREAT performance

[resample]

@pyro

I got the i7 non vpro version

I genuinely hope the IOMMU groupings are exactly the same across all the three versions.

If there’s somebody who owns a Framework with one of the other two versions could post their IOMMU groupings here, I’d really appreciate it!

[Somebody]

Thanks for posting this @pyro. Do you know what kind of performance hits are expected with this setup? I have a 10xx series GPU as well, and was wondering if it was an acceptable hit. Do you have any recommendations for an enclosure? I am guessing virtualization is maybe few %s, and eGPU over TB3 speeds to have more of an impact… but unsure.

Back in the late 90s… When I was bored teen, I used to download random software from freshmeat.net or sourceforge.net and just run it… oh how things have changed.

[pyro]

@Somebody

Do you know what kind of performance hits are expected with this setup? I have a 10xx series GPU as well, and was wondering if it was an acceptable hit. Do you have any recommendations for an enclosure? I am guessing virtualization is maybe few %s, and eGPU over TB3 speeds to have more of an impact… but unsure.

yeah so virtualizing it literally is less than a percent performance hit. For all intents and purposes its native performance. Thunderbolt 3 however does take a decent percentage from the performance of the card, I don’t know exact numbers but its not insignificant.

[2disbetter]

@Frosty

What game got pirated to this extent? I’ve never heard of any game dev that got their game pirated so much they went out of business. Look at KSP, no DRM, no protections at all, I bought it once and have the game files and could easily transplant it into another system. The population of people who don’t pay for games is miniscule, frankly. DRM was not lobbied for by small developers, it was pushed by huge studios who absolutely were not looking out for the little guy.

and yet…

@2disbetter

Nah there is no excuse for piracy. There is no game so important that stealing it is warranted. It always baffles me how people so easily justify jibbing software developers without a second doubt, but would never think of getting their car repaired and just driving off before paying the bill. (not saying you do this, just a comment on the nature of piracy in general)

How much easier would it have been to argue against DRM if piracy was never a thing? I don’t like DRM at all. What I’ve paid for should be mine without reservation. But trying to pretend that piracy doesn’t matter because you don’t see the negative affect is not a solid argument.

[CraigO]

@pyro

What hypervisor are you using? Boxes?

KVM with the virt-manager frontend, makes things easy, you can even edit the raw xml for the couple of things you need to through the gui

@pyro Thanks for explaining your setup. I’ve another ~week before my lappy hits and I plan to move away from dual booting in favor of KVM.

My experience w/KVM is… casual at best. Apart from google-fu, do you know of a good reference/resource to help me understand how to dial in a Winderz vm under KVM? tia

[CraigO]

Update: found the following (very lucid) article on setting up Windows w/KVM. Now, all I need is the lappy…

How To Use Linux KVM To Optimize Your Windows 10 Virtual Machine - Front Page...

[gs1]

Finally a KVM tutorial in plain english! Will try this when my laptop arrives too…

[Enjewneer]

I don’t know what distro you plan on running, but I have followed this guide from the Arch wiki in the past with a TB3 laptop and eGPU with great success. May not be as plain English as you’re looking for, but it does get you up and running pretty quickly but also dives deep into how to optimize performance.

https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF

[CraigO]

@Enjewneer

I don’t know what distro you plan on running,

Ubuntu but I’m warming up to rolling releases like Arch. But, hey, they’re related closely enough that this’ll help. Much thanks.

[2disbetter]

My question would be, how does the VM handle not having the eGPU there to passthrough? Will it automatically default to using a shared intel graphics chip?

Or will the VM not boot if the eGPU is not present?

[pyro]

The vm will not boot if the egpu isn’t present, for this I usually set up 2 windows 10 vms that use the same virtual hard drive, one with the device attached, and one with the virtual you and spice display.

[Paal_Joachim_Romdahl]

Hey

Does anyone have an update on this? Newbie here…
I am looking into getting a framework laptop in about 4-5 months and it would be great to be able to run Windows, OSX and Linux apps all on one laptop. Through whatever hopefully straight forward ways that are possible. Thanks!

[CraigO]

i7-1165G7 + 1TB - SN750 NVMe + 32GB DDR4-3200
Ubuntu 21.10 & Winderz 10

Followed the above instructions - no go. Windows performance was OK. Running a 15 y.o. game brought it to its knees. I’ve gone back to dual-boot.

(Btw, the idea of 2 Windows VMs to manage an eGPU is more than I’m willing to admin.)

[Mdaly001]

@pyro would you be willing to share some of how you got it working?

Specifically what pci ids did u pass thru?

Are you using kernel arguments with the vendor Id? I tried driverctl and it kept failing

Thanks in advance!

[pyro]

@Mdaly001

Specifically what pci ids did u pass thru?

Are you using kernel arguments with the vendor Id? I tried driverctl and it kept failing

Thanks in advance!

sorry for the late reply to this, the PCI-ids will change from card to card so sharing mine won’t help you any. I went the kernel arguments.

I just followed this guide. one thing to note is the pci-ids are the things right by the names of the devices. for example 0b:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] [10de:1b06] (rev a1) 0b:00.1 Audio device [0403]: NVIDIA Corporation GP102 HDMI Audio Controller [10de:10ef] (rev a1) the pci-id is 10de:10ef and NOT 0b:00.1

also you will have to pass the WHOLE IOMMU group like that, not just the video device, but the audio device and the pci bridge in the same group as well.

so my kernel parameters are intel_iommu=on iommu=pt vfio-pci.ids=10de:1b06,10de:10ef on my desktop.

I’m using systemd-boot instead of grub or anything like that so my arch.conf loader files looks like this:

title	Arch Linux
linux 	/vmlinuz-linux
initrd	/initramfs-linux.img
options	root=PARTUUID=52268f2f-d6d9-e542-998f-4ca664ce0010 rw intel_iommu=on iommu=pt vfio-pci.ids=10de:1b06,10de:10ef

[wkos]

@pyro Have you tried passing through the integrated Xe graphics to a guest yet?
On my desktop with 1 (dedicated) rtx gpu this works fine.
The GPU just goes to black after Grub and is never used by my Arch host and starts outputting once one of the guests starts.

I’m hoping to do the same on my Framework lapop once it arrives without needing to use an eGPU perse while i’m on the road. I really like the separation/isolation using multiple vm’s gives me on my desktop, but still retain the bare metal feel of using direct output of the GPU instead of VNC/RDP.

[pyro]

I have not tried it with the integrated GPU, doing like single GPU passthrough I’m guessing is what you’re talking about. This should work pretty well, it will just kill xorg/Wayland while you’re booted into a vm as there will be no GPU left to run the host, si it will essentially be alot like dual booting, only able to use one system at a time.

Honestly I’ve actually abandoned doing GPU passthrough for gaming, proton/wine works great now, take a bit of fiddling sometimes but not often, and with the way anticheat works on Linux it has less access to your system than it does on windows. Plus with the steam deck being as popular as it is proton/wine is a valid target for devs, most of which while not officially supporting it (as in like providing technical support to Linux users) have at least enabled the anticheat to work in proton/wine. The only one I personally want to play but can’t right now is halo infinite… But I don’t even really want to play that too bad, only cause some of my friends play it sometimes.