Anyone have any luck getting ollama to work in Windows? I’m getting no GPU utilization
1 Like
I am running into the same with Fedora 42 and Ubuntu 24.04.
CPU utilizaiton is kicking in and giving me about 26 tps but not seeing it use the GPU side of things.
This is even with the amd drivers installed.
When I installed ollama, I did see it pull the files needed for rocm but not sure if it is actually being used or not.
1 Like
I don’t have a Desktop yet but I did see this some time ago. There’s also this discussion on the Desktop Community. Check the pointers from @lhl in particular.
And I just noticed this, too. Try the Discord group.
I hope they help. Let the forum know how you go, please?
LM Studio works out of the box. I believe Ollama was working with ROCm acceleration at one point, but its possible that regressed.
2 Likes
ROCm has to be built specifically for any GPU - by AMD.
ROCm support is tied to the GPU’s GFX IP (architecture version) and whether AMD has enabled/whitelisted that PCI ID in the ROCm runtime.
-
Example: gfx1100/1101 (RDNA3 dGPUs) → enabled.
-
gfx11.5 (Strix Halo iGPU) → not enabled, so ROCm won’t load.
It’s not just “compile once, runs everywhere” — AMD must add that GFX target + device ID into ROCm builds.
1 Like
Unfortunately from what I’ve seen it doesn’t integrate nicely with Home assistant like Ollama does, although maybe I’m wrong!
I hope they fix it, because running it locally for Home Assistant voice control is what I am interested in. Like…. Jarvis 
I just tried LM Studio on my Framework Desktop Windows 11 Pro and, unfortunately, it appears that the latest Vulkan V1.52.0 driver doesn’t really use the iGPU. @nrp do you have any info about how you setup LM Studio?
GitHub - oobabooga/text-generation-webui: The definitive Web UI for local AI, with powerful features and easy setup. does work for me using Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf. I saw around 40 tokens/second. My colleague on whatever Mac he has sees about 52 tokens/second so I felt a tad bummed.
Install older version of LM Studio and make sure you have disabled the Runtime updates in settings. Or check the runtime settings → versions and uninstall the 1.52 version. This depends which version of LM Studio you first installed, the newest versions don’t have the 1.50 version of vulkan llama available.
Then make sure Vulkan LLama is on version 1.50 (not 1.52). 1.52 is busted and won’t use the iGPU fully.
2 Likes
Shouldn’t be bummed. Mac silicon is freaking awesome…. and locked down tighter than a bank on Sunday. Linux on anything else gives so much more freedom to do things the way I want, including learning more, too. I enjoy my MacBook Pro for it’s reliability for boring stuff. But Linux is liberating, even if not yet to Mac silicon speed.
1 Like