Qwen3.5 4B

on AMD Strix Halo 128GB · llama.cpp b8171 / c17dce4f / Vulkan RADV

Oct 14, 2026

Throughput

28.0 t/s gen · 1259.1 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

UD-Q8_K_XL (GGUF)

System RAM

128 GB

Summary

User reports Qwen3.5-4B-UD-Q8_K_XL at 28 t/s generation and 1259.1 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp b8171 with the Vulkan backend, ngl 99, and the UD-Q8_K_XL quant; the native table lists type Q8_0, 4.21 B parameters and 5.53 GiB. The rates are separate synthetic phases, PP512 at 1259.14 +/- 3.82 t/s and TG128 at 27.95 +/- 0.07 t/s, not a same-request prompt/output pair.