It almost looks like AMD has provided some code for using Ryzen AI on Linux:
Has anyone looked into that yet? Maybe I’m just seeing things.
1 Like
Not very interesting. It’s only for model compilation, not for actually running inference.
Ryzen AI 1.5 will have some more linux inference stuff.
Has anyone benchmarked the Ryzen AI processors to see how many tokens per second they can achieve on Linux without AMD properly supporting the NPU?
I think for people considered doing AI-related work, this would be an important detail.
There’s still no NPU support for Linux.
This feels like one of those situations where maybe a company tells its investors “we really care about AI!” but the facts on the ground show the executives don’t really care. At least not in the sense that AI enthusiasts do.
What do you mean? In specific use cases? All the components are documented and point to appropriate repositories.
You’re right, the NPU driver is available, documented, and fully loaded under Linux.
What I meant is that there are currently no practical applications on Linux that can leverage the NPU for inference. For example, AMD has some projects in this area that focus on NPU inference.
- GitHub - amd/RyzenAI-SW: AMD Ryzen™ AI Software includes the tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzen™ AI powered PCs.
- GitHub - amd/gaia: Run LLM Agents on Ryzen AI PCs in Minutes
But that’s Windows-only, not Linux.
It seems we’ve been waiting quite a while for AMD to deliver proper Linux NPU inference support. That’s why I said experimenting with the NPU isn’t really feasible just yet.
Ah I see. Yes it’s more oriented around a developer flow right now.
Although looking at the links you shared GAIA is supposed to have Linux support as well.
But I see the other link that lemonade isn’t ready. Got it now.