Llama.cpp/vLLM Toolboxes for LLM inference on Strix Halo

[Djip]

Compiling VLLM from source on Strix Halo

Finally put together a recipe for compiling VLLM from source on Strix Halo. Please note that while it worked yesterday, it may stop working tomorrow if they introduce any breaking changes. Please let me know. Building vLLM for AMD Strix Halo This is a recipe tested on Fedora 43 beta, but should work in any other recent Linux distro. Prepare environment First, install uv if you don’t have it on your system yet: Installation | uv I prefer the pipx route, but you can use any method that works. A…