Not sure if I’m doing everything right – I’ve freshly installed Fedora 42, downloaded the latest LM Studio, and installed some models – specifically Qwen3 30B, and gpt-oss 120b. I’m getting the same tokens per second regardless if I use full GPU offload with the Vulkan llama.cpp engine, or if I use the CPU only llama.cpp engine.
For gpt-oss, I get about 20 tokens/sec, and Qwen3 coder 30b a3b I get about 30 tokens/sec. Again, with the Vulkan or the CPU only backend.
Also I’ve tried setting the ttm pages_limit and page_size_pool kernel parameters to 26214400, also tried with them not set and setting the video ram to 96 GB in the bios, etc. Same performance.
Also, even though everything says it should be on GPU with the vulkan backend, top command shows high cpu usage by lm studio processes. So it is acting like it is offloading most layers to CPU anyway.
How can I tell if lm studio is actually using the iGPU?
Same here, on Fedora 42. I’m not an expert on this so I don’t know if I am doing something wrong in LM studio or anywhere else. And have many questions, probably poorly formulated because of my ignorance.
I played with ollama before on an old rig with 24gb radeon card (don’t remember the model right now), it was difficult to make it work but it worked.
My intended use case is mainly for programming assistance
I understand that Ollama doesn’t support this APU so I’m open to learn lm-studio or any other tool that gets me there.
But my first questions are mostly about things before choosing the tool to run the llms
Should we change BIOS settings or not for (VRAM assignment) lm studio to use the iGPU
What tool can we use to assess wheter it’s using iGPU ot not. I actually don’t know if it’s doing it radeontop seems to say no, Mission Center (installed through flatpak) seems to say no also.
Should we play with kernel parameters on boot and which parameters/values?
Should we try using the last version of ROCm?
lm studio says there is only 42GB available, is this number reliable? Is there away to go beyond 96GB on linux as some say?
I’d love an in depth tutorial for noobs on how to configure this.
Asking myself the same questions. I’m on Fedora 42 as well. Also seeing 42GB VRAM in the LM Studio settings. Thing is, gpt-oss 120b is about 60GB and runs very well. Which is good! But I would also like to understand LM Studio a little bit better and am also interested in manually allocating VRAM. Given the Framework blog post some weeks ago, this all works already great on Windows. Another guide for LM Studio + Linux is also teased within this article. Looking forward to it and/or some insights from other community members.
amdgpu_top --smi is the way to go - it will give you a compact view of temps and memory usage and a list of processes using your GPU w/ percentage usage per process. Just as an FYI, radeontop hasn’t been updated for years and while it nominally works… I wouldn’t recommend it.
For those looking for the easiest way to get started, a small but dedicated AMD team supports https://lemonade-server.ai/ - not only should it be straightforward to get inferencing w/ known good configurations, but they’re also active on their Discord to give support.
I’d suggest using llama.cpp directly instead of LM Studio. You will have more granular control over parameters and latest build (LM Studio usually lags behind and doesn’t expose all useful stuff via UI).
Just run llama-server and it will provide OpenAI compatible API endpoint and basic, but functional chat UI.
SOLD! Lemonade Server looks awesome. I’ve been using ollama on a Ryzen system and Nvidia 3060 12G, Open WebUI on the frontend. Model management was awkward. I really like how the user experience looks with OL. I’ll start with that on my new Desktop.
6 hours ago I installed gpt-oss 120b in LM Studio (Vulcan engine) and it was utilizing the GPU, btop showed that 60G of VRAM were taken. The performance was ~30-40 tps
I come home from work and try again, but this time the model is not loaded into VRAM and it seems to be stuck in CPU/ram only mode, the performance is ~10 tps.
I have no idea why or how… I tried to fix it but no luck, it’s still not using the GPU
Arch with the latest packages, opencl-amd installed (v7.0.0)
I seemed to recall seeing it use the GPU the first time/s when I first installed the system a couple of days ago but I was doubting myself because no matter the model or the settings it has kept using just the CPU since this morning.
Now I remember letting it update the runtimes, and I see that’s the problem.
i’ll probably end up using llama.cpp in the end but lm studio is very nice for testing the models and the basic parameters
Same here, that solved my problem too. And here I thought I was hallucinating when I remembered 45+ tokens per second on gpt-oss 120b.
So I figured out how to roll back to an older downloaded version, but for others – how do you browse / download a specific version from LM Studio’s web site, or via the tool, if you don’t currently have it?
That’s actually a very good question. If it happenesd once it can happen again.
On the LM studio GUI, on Mission control > Runtime, on the left side:
In the Selections tab you can choose 1.50.2 which works and I also deactivated both the Auto-update selected Runtime Extension Packs and Auto-delete least recently used Runtime Extension Packs (just in case)
Also you can select the All or My Engines tab and clicking on the three dots of the vulkan llama.cpp you can Manage Versions and delete the offending ones.
But I couldn’t find a place where it lets you download any previous version so if you installed after 1.51 vulkan llama.cpp was included in the AppImage you would be stuck with non working versions until they release a fixed one?
I don’t have anything against LM Studio (vs Ollama, who I’m not a fan of), but I will mention if you have like build-essentials or base-build or basic gcc/cmake etc installed, building your own llama.cpp really is probably the way to go, it’s incredibly simple and you have full control over which build revision you want: llama.cpp/docs/build.md at master · ggml-org/llama.cpp · GitHub (git clone, cmake config && cmake build, takes about 1min w/ a 395 to build).
As @Eugr mentioned, it auto-builds llama-server, an OpenAI compatible endpoint server. For GUI, the slickest/easiest one I’ve found is https://msty.ai/ (for open source there’s Jan.ai and some others).
Thanks, lhl, I will try that next, because in the end I want to use the framework desktop as a server for inference.
I received my framework desktop 2 days ago and was eager to test it out, lm studio seemed the easiest path and I got trapped into this.
But I’ll definitely try llama.cpp, also because I prefer to use OSS as much as possible.
I got LM studio to use the GPU just fine, when setting the GPU offload to the max. However no matter what GTT setting, it refused to use more than 50% of the RAM for the iGPU. Any models larger than that slowed down to a crawl, took forever to load and even longer to run.
So, as long as you are fine with staying below 50% of your memory, LM studio is fine. But I think that is a waste of the system. With llama.cpp it was no problem to use the GTT to the very limit. Even 61GB for the iGPU, with only the scraps left for CPU works surprisingly well, enabling the use of even GPT-OSS-120B on my 64GB Framework Desktop.
If LM Studio is using the ROCm backend, then you need to disable mmap. There are a few knobs you can additional tune to improve llama.cpp (and llama.cpp derivatives like LM Studio, Ollama) to perform better: llamacpp-performance – Strix Halo HomeLab
Agreed. I don’t know about Vulkan/Rocm builds as I don’t have the hardware (yet), but even for NVidia you want to build it yourself as they don’t include support for additional KV quants (q5_1, for example) in the standard builds. Not that a big deal for unified memory systems, but useful for VRAM-constrained systems as you can fit more context.
