llamaperf
Oct 12, 2026
Throughput
12.0 t/s gen
Quant
UD-Q4_K_XL (GGUF)
KV cache
F16
Flash Attention
on
System RAM
128 GB

Summary

User reports Qwen3.8-Flash-Next at 12 t/s generation on an M4 Max 128GB, with the source's 128K benchmark point given as 11.98 tokens/s. Setup is llama.cpp with UD-Q4_K_XL and f16 K/V cache, all layers offloaded, flash attention on, batch 1024 and microbatch 256. The figure is generation after cache prefilling, not prompt processing or TTFT; the source's f16 KV series is 16.62 t/s at 64K, 14.93 at 81K and 11.98 at 128K.