Anyone utilized the NPU?

Framework Desktop
5

I did find out that there are LOTS of information on this. There is an IN KERNEL driver that I managed to get up in fedora, but its QUITE difficult to get doing in linux. There also seems to be a lot of ‘dueling banjos’ in the form of userspace firmware vs. in kernel firmware. I am looking at trying the dkms version, just to see if I can get things giong. I will post a partial guide here when I get on my framework desktop. This guide is NOT COMPLETELY WORKING would love some help on this!

NPU & DynaGuard on Strix Halo

I found a very nice model that allows me to load a ‘guard’ model using the NPU on my strix halo. This frees up the gpu from having to process the guard model, which loads on the NPU. This document is for fedora core 43 and uses toolbox for the npu part.

Getting the NPU working inside a toolbox container

I prefer to keep this isolated from the main system, since I have to add all kinds of strange repos that I don’t want polluting my main system. The Gemini and I came up with the following: Which lets you get a working environment to access the NPU. Ensure your user is in the ‘render’ group as this is what the npu uses.

 # toolbox create ryzen-ai -- --device /dev/accel/accel0:/dev/accel/accel0 --group-add keep-groups
 # toolbox enter ryzen-ai
 # Check on the status of the driver
 stat /dev/accel/accel0
 ####Outpu Follows
 File: /dev/accel/accel0
   Size: 0           Blocks: 0          IO Block: 4096   character special file
 Device: 0,6 Inode: 529         Links: 1     Device type: 261,0
 Access: (0666/crw-rw-rw-)  Uid: (65534/  nobody)   Gid: (65534/  nobody)
 Access: 2025-12-24 20:23:59.532992109 -0500
 Modify: 2025-12-24 20:23:59.532992109 -0500
 Change: 2025-12-24 20:23:59.532992109 -0500
  Birth: 2025-12-24 20:22:52.187999977 -0500
 ​

If you see something like the above with no errors, things are working. Basically this puts through the npu with other having read/write access to the device.

Getting the necessary tools to load the models using the XRT tools from AMD

These commands install and eventually correct the library problems that come up with fedora core 43 and the npu userspace tools.

 dnf install -y xrt xdna-driver tcsh
 sudo dnf copr enable xanderlent/amd-npu-driver -y
 sudo dnf install -y xrt xdna-driver tcsh
 # Create a symlink so that 'lib' points directly to 'lib64'
 # This is the cleanest way to fix all XRT library pathing issues at once
 sudo ln -s /usr/xrt/lib64 /usr/xrt/lib
 # Update the linker cache
 ldconfig
 ​

It is CRITICAL that you have IOMMU enabled. It caused me no end of pain. Won’t work in the container OR the main system. To adjust, edit grub.cfg thusly

 vim /etc/default/grub
 GRUB_CMDLINE_LINUX="rhgb quiet amd_iommu=on iommu=pt amdgpu.gttsize=131072 ttm.pages_limit=33554432"
 ####
 sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Just make your cmdline match this the amd_iommu=on is CRITICAL

Also the command to check things may not work too well these fix it:

 sudo ln -sf /usr/xrt/lib/libxrt_driver_xdna.so /usr/xrt/lib/libxrt_plugin_xdna.so
 sudo ln -sf /usr/xrt/lib/libxrt_driver_xdna.so /usr/lib64/libxrt_plugin_xdna.so

To check if everything is happy do the commands below:

 /usr/xrt/bin/xrt-smi examine
 ##remaining is what success looks like!
 System Configuration
   OS Name              : Linux
   Release              : 6.17.12-300.fc43.x86_64
   Machine              : x86_64
   CPU Cores            : 32
   Memory               : 128078 MB
   Distribution         : Fedora Linux 43 (KDE Plasma Desktop Edition)
   GLIBC                : 2.42
   Model                : Desktop (AMD Ryzen AI Max 300 Series)
   BIOS Vendor          : INSYDE Corp.
   BIOS Version         : 03.03
 ​
 XRT
   Version              : 2.19.0
   Branch               : 
   Hash                 : 
   Hash Date            : 2025-04-25 00:00:00
   virtio-pci           : unknown, unknown
   amdxdna              : unknown, unknown
   NPU Firmware Version : 1.0.0.166
 ​
 Device(s) Present
 |BDF             |Name          |
 |----------------|--------------|
 |[0000:c4:00.1]  |RyzenAI-npu5  |

If it looks like above you are ready to proceed with remaining steps. At SOME point I might automate these or let the AI do it for me!

Not quite! More annoyances!

 # make the drivers load our firmware
 cd /usr/lib/firmware/amdnpu/17f0_10/
 sudo xz -d -k npu.sbin.xz
 ​

I got the drivers loaded, but the xrt checks failed because of a mismatch. I also need to convert the dynaguard model to a format that the npu can consume.

1 Like