Compiling VLLM from source on Strix Halo

Framework Desktop
22

Any way to get MTP(multiple token prediciton) work, with vllm/rocm?

23

Haven’t tried yet, so no idea.

24

Got MTP work. TPS dropped sharply from 15 tps to 8 tps.

25

Yeah, I tried it on my Spark too and it dropped from 43 t/s to 30 t/s or so.