Using Incus on Debian 13 with A hybrid kernel a Fun primer

Framework Desktop
1

I won’t be fully documenting everything here but I have QUITE a nice setup for labs/experiments here.

  1. I use debian 13 USERLAND
  2. I replace debian USERLAND with zabbix kernel
  3. I have a custom power script (basically it uses herustics if gpu goes up to 80% for more than 5 secnods we switch from ‘balanced’ to ‘perfomance’ It stays on performance for 300 seconds and then goes down. So my “MIN” power draw is 5 watts (gpu) and MAX can go upt platform madx (140 or so) in high performance scenaris.
  4. The hybrid kernel also loads the proper drivers for the NPU (you need to enable iommu)
  5. I load my ‘test’ models into LXC unprivilged containers with ‘mapping’ to enable local access
  6. I also have a zed container
  7. I use opencode as a harness
  8. I did use spensive (but free for a while) openai codex to build a lot of the studd.
  9. This hybrid setup gives me STABILITY of debian userpace with “RECENTNESS” of zabbix kernel
  10. Finally, I use the ‘whatever’ image to run vllm, (rarely), llama.cpp (various models mostly QWEN 36B MTP) MTP IS THE BOMB
  11. You can run qwen 27b dense, it follows directions well, but is like 3x slower, (sadness).
  12. I also have a ‘models’ mount point (read only) profile, an ‘internal’ containers network, and a ‘macvlan’ network option for local access (the machine itself can’t access macvlan, which is whny I have the 2 network profiles0.
  13. Incus has a nice ‘docker’ importer, so I just manufacture dockerfiles with my various lab experiments, and pull from the local repo.
  14. I do write MOST of my mcp’s in NIM (I made a nim MCP with a more advanced model so the ‘lesser’ models can actually code competently.