Mistral-8x22b (source label; Mixtral family reference) / Q4_K_M / Vulkan native PP512 TG128 / runtime build unknown
AMD Strix Halo 128GB · llama.cpp / Vulkan KHR cooperative matrices / build unknown
- generation:
- 8.9 tokens/s
- prompt processing (prefill):
- 106.0 tokens/s
- quant:
- Q4_K_M
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Mistral-8x22b at 8.9 t/s generation and 106 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp with Vulkan on the Radeon 8060S iGPU, Q4_K_M quant, 6 threads, and maximum GPU offload (-ngl 999). The source gives 8.92 t/s and 106 t/s for the separate PP512 and TG128 phases, rounded here to 8.9 t/s; the two are not an end-to-end rate.