Framework Mainboard with Ryzen AI Max 385 for light AI

[Mango76]

Hi!

I have seen a lot of topics about building custom-built mini PCs using the Framework Strix Halo mainboards.

I am not sure if this has been discussed in the past but I am planning to purchase a FW Desktop mainboard with a Ryzen AI Max 385 for light AI use. I am thinking of running small models like gpt-oss 20 B on it. Would you say that it is possible to run such models at decent speed given that the Strix Halo series has seen a lot of software/driver improvements when it comes to AI?

Then I would like to know if someone has build a PC using the Jonsbo N10 already. Because I am looking into buying that one.

At last, what are your experiences with Framework Desktop mainboards? Have many of you run into problems with curl whine or mainboard failure issues?

Any answer is really appreciated. Thank you in advance.

Kind regards

M.

[Squiggler]

First item:

I am planning to purchase a FW Desktop mainboard with a Ryzen AI Max 385 for light AI use.

What kind of AI work are you looking to do? My interest is code generation, and I’ve a thread on that topic here:

How's your real-world AI (code generation) on the Framework Desktop?

Hey Jeffrey, Good news — ROCm does support Strix Halo (gfx1151), starting from ROCm 7.0.2. Setup: 1. Install ROCm 7.2 via your distro’s native package manager. 2. Set `HSA_OVERRIDE_GFX_VERSION=11.5.1` and `HSA_ENABLE_SDMA=0`. 3. Verify with `rocminfo | grep gfx1151`. For Ollama, add both as `Environment=` lines in the systemd unit file. The [hogeheer499-commits/strix-halo-guide]( GitHub - hogeheer499-commits/strix-halo-guide: AMD Strix Halo local LLM guide: direct 100.0 t/s 30B Qwen MoE on…

Second item:

I would like to know if someone has build a PC using the Jonsbo N10

Various cases, including the Jonsbo, get a mention here:

For MB purchasers: what case are you getting?

I was getting the original framework desktop and I heard there was some noise issues. I didn’t feel like dealing with the potential problem because I do audio production. I figured I’d just get the main board and have some options with respect to Case and power supply. I’m currently looking at the fractal Terra because I like the aesthetic, but I’m also interested in thinner cases and doing some research on my options there as well.

Final item:

what are your experiences with Framework Desktop mainboards? Have many of you run into problems with curl whine or mainboard failure issues?

Failure is hard for each person to comment on, because only Framework see numbers to a statistically significant degree. Everyone else sees a handful of failures at worst, which is not enough to comment on. Your main guard against problems is the warranty, which is one, two, or three years long depending on where you live. I’d love to see returns data from Framework, but I assume it is regarded as proprietary information by any system maker.

[Squiggler]

I assume you’re interested in using a case to permit the addition of a GPU. If so, this thread will be useful:

What external GPUs work reliably in the x4 slot?

I’ve been incrementally researching the Framework Desktop for AI inference purposes, and have asked a couple of questions here, and have been going through a few conversations with AI. Slowly and iteratively, my understanding (of local AI and the Desktop’s capabilities) improves. My hope is that while I will subscribe to commercial AI providers (for code generation) for now, I would like to run my AI capability entirely myself in the future. I am pretty much sold on the Desktop, but would like …

You may find it useful to know that the PCIe interface is four lanes of data using version 4.0 of the protocol. This may not be a problem if you can fit models entirely in dGPU VRAM, since the additional delay is a one-off cost.

[Squiggler]

This forum contributor has a lot of video content and tooling to help with running AI on Strix Halo - highly recommended:

[convexproblem]

I have a 128gb 395 in a Soeyi U300, this is what it looks like at the moment (really horrible heatsink, waiting on my Framework heatsink, that’s a pleated prefilter at the bottom and the dust filter at the top to catch dust drifting downward while it’s off). Could’ve gone one size smaller on the case length if the idea is to stick to low profile air cooling.

I guess it depends on what you’re running but I would personally slam into the 32gb ceiling pretty fast with my habit of leaving 50 browser tabs open. The 8050 is not that much slower than the 8060. One idea for freeing 1-2gb of RAM is to add some extremely cheap old card like a Quadro P400 to run the desktop video output.

[Derrick_Chang]

From what I can see, gpt-oss-20b will fit nicely (~11 GB VRAM usage) and run quickly according to kyuz0’s benchmarks on github for the bigger brother AI Max+ 395 (Donato Capitella on Youtube, he makes guides).

The Max 385 and Max+ 395 have the same memory bandwidth of 256 GB/s, so the inference speed should be about the same (Memory Bandwidth bound). Here’s a single reddit post two years ago about the 7900xtx vs 7900xt, the Memory Bandwidth of the 7900xt is 83.3% of the 7900xtx and the llama.cpp inference speed is about 81.2%, implying correlation.

The Max 385 and Max+ 395 have 32 and 40 compute units, respectively. That means the prompt processing speed for the Max 385 should be about 80% of the Max+ 395. The same reddit post above shows that using 87.5% of the compute units gets 85.2% of the prompt processing speed, again implying correlation.

The same cannot be said of the two Nvidia cards shown, they’re different generations with enough notable changes.

So just mathing it out:

1. gpt-oss-20b with rocm-7_2_3
   1. AI Max+ 395 (from the kyuz0 benchmarks)
      1. Prompt processing (pp512): 1786.17 t/s
      2. Token generation (tg128): 73.05 t/s
   2. AI Max 385 (calculated)
      1. Prompt processing (pp512): 1428 t/s (80% of AI Max+ 395)
         1. Drop down to 1071 t/s if we expect 60% of the performance, still manageable.
      2. Token generation (tg128): 73.05 t/s (the same)