llamaperf

Mistral Medium-3.5-128B / Vulkan RADV native PP512/TG128

on AMD Strix Halo 128GB · llama.cpp / Vulkan RADV / b9029

AMD hardware
Oct 13, 2026
Throughput
2.9 t/s gen · 62.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
Flash Attention
on
System RAM
128 GB

Summary

User reports Mistral-Medium-3.5-128B at 2.9 t/s generation and 62 t/s prompt processing on an AMD Strix Halo 128GB system with a single Radeon 8060S iGPU. Setup is llama-bench with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention; the source lists a 120 GB soft GPU-memory cap. The 2.9 t/s is the RADV Vulkan figure; ROCm gave 2.92 t/s and AMDVLK 2.99 t/s, while the main table lists both Vulkan runs as 2.9. These are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request.