Had mine running since Tuesday. I’ve got Open-WebUI running in a Docker container and fronting a Vulkan-enabled llama.cpp container running Qwen3-Coder-30B that I interact with in Open-WebUI via Qwen-Agent (another container) to allow agentic RAG via a local SearxNG container and a QDrant vector database container for saving and searching through retrieved data.
Still not entirely happy with the RAG responses, but I’m fiddling with the system prompt to get it a bit more robust.
Getting about 43-45 tps with Qwen3 directly in Open-WebUI; Qwen-Agent makes everything a bit slower since the agent’s all Python.