Qwen3.8 27B
on AMD Strix Halo 128GB · llama.cpp · 32,768 ctx
Use cases
codingmathtool-useagentic
Summary
User reports Qwen 3.8 27B at 14.02 tok/s unassisted decode on an AMD Strix Halo (Ryzen AI Max+ 395, 128 GB).
Setup is a ROCmFPX-enabled llama.cpp fork with ROCmFP4_FAST quant (4.26 bpw, 13.55 GiB GGUF) and Asymmetric TurboQuant KV cache at 32K context.
With MTP speculative decoding, decode reaches 34.82 tok/s (2.84x) at greedy sampling; deep spec n7/p0.35 reaches 36.04 tok/s on JSON and code. At temperature 0.8 draft acceptance collapses to about 25% and speculative decoding returns close to the unassisted 14.02 tok/s.