[GUIDE] Use NPU (XDNA2) with Arch Linux and FastFlowLM!

[GreyXor]

Good News, Everyone!
We can now use the NPU for LLM inference with FastFlowLM. Here’s my little guide for you to achieve it

To use the NPU on Arch Linux you need some patches that will soon be merged into linux, xrt-plugin-amdxdna, and xrt.

You need to :

1. Build linux-git kernel package with patches
   2. Build xrt and xrt-plugin-amdxdna with patches
   3. Compile FastFlowLM or use my aur package

1. Build linux-git with patches
Usin this AUR package AUR (en) - linux-git you have to change the URL:

$_srcname::git+https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux

become

$_srcname::git+https://gitlab.freedesktop.org/drm/misc/kernel/#branch=drm-misc-fixes

Then makepkg -si

2. Build xrt and xrt-plugin-amdxdna with patches

~~ https://gitlab.archlinux.org/superm1/xrt/-/tree/update-version~~
~~https://gitlab.archlinux.org/superm1/xrt-plugin-amdxdna/-/tree/update-version~~

On each repo you have to execute makepkg -si

3. Compile FastFlowLM
Follow this link to build FastFlowLM
~~

GitHub - FastFlowLM/FastFlowLM: Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just...

You can also use my aur package https://aur.archlinux.org/packages/fastflowlm-git~~

You may also need to edit /etc/systemd/system.conf and change #DefaultLimitMEMLOCK to DefaultLimitMEMLOCK=infinity

Happy inference!

Special thanks to @Mario_Limonciello

[SneakyPM]

Very interesting. Can any GGUF model from HuggingFace be used ?

How does it compare to iGPU performance wise tg and pp ?

Can we run iGPU+NPU with NPU for prompt processing and iGPU for text generation ?

[GreyXor]

1. No, they need to adept model to make them work with NPU. List of available model’s here: Models · FastFlowLM they also making tools to automate conversion of GGUF to NPU-compatible
2. There’s some benchmarks here: Benchmarks · FastFlowLM
3. Not with FastFlowLM which is NPU-only, at least for now.

[Shiroudan]

I suppose this will not work on the AMD 7040 mainboards considering the repo seems to require XDNA2?

Thank you for the detailed guide however!

[George_Sofianos]

For people that don’t want to build the Linux kernel, I’ve pushed a dkms package that should work with kernel 6.18+. Here is me testing the NPU in Lemonade

[John_Scott1]

This is awesome stuff guys, thanks for sharing. For Ubuntu, AMD now have a PPA for the xdna2 DKMS package. There is some more info at FastFlowLM/docs/linux-getting-started.md at main · FastFlowLM/FastFlowLM · GitHub - that doc refers to a .deb package for fastflowlm which doesn’t exist yet but I’m sure it’s only a matter of time.

I was able to build FastFlowLM from source and after a small amount of fiddling around (had to make a symlink from my build directory to src/xclbins) now have gpt-oss-20b running at 18 tok/s via the NPU, even with my CPU in power-saving mode. Very cool.

[Guest383]

Thank you. I had to install your amdxdna-dkms package to get it to work on CachyOS because Cachy’s firmware was newer than the driver’s version or something. PSA: Do not set amd_iommu=off as a kernel boot parameter if you want the NPU to work.

[Caleb_Maclennan]

I think you can scratch off step 3 from the list too, this seems to work out of the box with the fastflowlm version packaged in stable Arch repos now: 0.9.35. I got it to work without -git package.

For reference I also didn’t use the linux-git kernel, I use the linux-mainline AUR package that is 7.0.0-rc4.

[GreyXor]

Sure thing, thanks!
I will also update it completely when Linux 7.0 will be released(~2026-04-19)

[J_Dog]

I just upgraded to Ubuntu 26.04 on my HX 370. All the drivers are preinstalled and the NPU just works after installing flm! Easiest way to get started is to install the lemonade server and flm. Instruction here. The instructions basically say:

• Install the Ubuntu 26.04 FLM deb from GitHub.
• Install the Lemonade server with

sudo add-apt-repository ppa:lemonade-team/stable
sudo apt install lemonade-server

After that, the lemonade webapp should show a bunch of models under FastFlowLM NPU that you can download and use.

[adb-sh]

for the nix folks among us. i found these two flakes:

• GitHub - robcohen/nix-amd-npu: Nix flake for AMD Ryzen AI NPU (XRT + XDNA driver) · GitHub
  • builds xrt and the xdna driver
  • there is also a package for fastflowlm
  • worked for me
• GitHub - noamsto/nix-amd-ai: AMD AI inference stack for NixOS (XRT, XDNA plugin, FastFlowLM, Lemonade) · GitHub
  • seems to be based on the first repo
  • also builds lemonade with die fastflowlm, ROCm and Vulkan backend
  • sadly the included xrt version seems to be incompatible with my hardware (AMD Ryzen AI 9 HX 370) or maybe its just not correctly presented to the userspace - might investigate this further

[Josh_Hubert]

If anyone on arch (I’m on cachyos) is using flm with lemonade, lemonade integration is broken for the current pacman version of flm. You have to build the previous version v0.9.38 from source.

Steps:

# reiteration of fastflowlm deps (LLM on NPU support)
#sudo pacman -S xrt xrt-plugin-amdxdna
# update memlock settings and log out to let it take effect
#echo -e "*                soft    memlock         unlimited\n*                hard    memlock         unlimited\n" | sudo tee -a /etc/security/limits.conf

# remove broken:
sudo pacman -R fastflowlm
# build flm v0.9.38 from src instead
sudo pacman -S cmake ninja boost
cd ~
git clone --recursive https://github.com/FastFlowLM/FastFlowLM.git
git checkout v0.9.38
cd FastFlowLM/src
cmake --preset linux-default
cmake --build build
sudo cmake --install build
flm -v