I have been exploring the features of my FW16 (AI 9 HX 370) and it turns out that AMD’s local‑AI software stack (Lemonade) only reaches its full performance when both the CPU/NPU and GPU are AMD. So my fancy shmancy nvidia card doesn’t help here. But the only Radeon option for the Framework 16 today is still the RX 7700S.
AMD co-develops an open-source local LLM server called Lemonade. On Ryzen AI 300/400-series boards (every current Framework 16 mainboard, NVIDIA module or not), Lemonade can split the work between the GPU and the NPU, like by doing prompt processing on the XDNA2 NPU and token generation on a GPU. AMD’s benchmarks show this hybrid split gets significantly faster performance than GPU-only. Supported discrete GPUs are the RX 6000/7000/9000 series.
BUT hybrid mode is AMD-exclusive.
Put an NVIDIA card in and Lemonade falls back to CUDA-only. It works, but it means the NPU built into every one of these mainboards regardless of which GPU module you buy just sits idle.
So right now the choice is: RTX 5070 for newer/faster GPU but with an NPU that never does anything, or RX 7700S to keep the NPU relevant but be stuck on RDNA3 while NVIDIA buyers get the newer part.
This is a massive architectural problem with the FW16 that I didn’t even anticipate when I bought into the platform. It feels like an unfair abuse of high end components at a time when components are overly expensive and hard to come by.
I would like to humbly request a newer/nicer Radeon module option for the FW16. Please pretty pretty please with a cherry on top.
You can run quite a few models on just the NPU, which you might want to do even when you have a GPU available, to save power/improve battery life. Plenty of developers have come up with interesting uses for even older and smaller NPUs, too. Your hardware would be more than capable of recognizing objects or gestures, for example, without the need for an LLM at all.
In any case, the 7000 series is actually the newest AMD discrete mobile gpu generation available… I’m fairly confident we’d see a Framework module for it if a newer series was released.
But if the 5070 doesn’t fit your use case, why not try trading for a 7700S? If you’re into the AMD AI ecosystem, but not gaming, you might prefer it, and I’m sure you could find a gamer happy to pay you for an upgrade.
I had no idea there was a hybrid mode available! I have tried looking into this (asking Google AI) and found nothing. I’ve been running solely on my RX 7700S for my AI stuff I’ve been testing and have been irritated that my 7840HS NPU has been a complete waste of hardware. I’d love a RTX 5070 especially if it’s 12GB (I mostly game)! Unfortunately the price has been too steep for me.
I have:
xrt_plugin-amdxdna 2.26: the XDNA driver plugin for XRT
xrt-base / xrt-npu 2.26.0: the XRT runtime stack
I installed them using locally built debs according to the amd/xdna-driver’s build process. The userland piece you need comes from the repo.
Unfortunately it’s only supported on arch and ubuntu right now. I still run ubuntu though and if I jump ship it will probably be to arch, so it works for me.
It’s true that the amdxdna.ko kernel driver was mainlined and has shipped in-tree since kernel 6.14, so Ubuntu 25.04 and later.
But the “userland piece” I mentioned isn’t the kernel driver, it’s the XRT SHIM library (xrt_driver_xdna.so) plus firmware, which come from the AMD xdna-driver repo’s build and get installed as separate DEB packages. None of that ships as part of the kernel or on default Ubuntu. So you still have to clone xdna-driver and build/install those packages yourself (at least at present).
I respectfully request a similar fix. I have the NVIDIA dGPU and I would love to be able to use BOTH my NPU/iGPU AND my NVIDIA dGPU. I’ve been trying to figure out how to do this and haven’t figured it out yet.