llamaperf
Sep 27, 2026
Throughput
31.7-35.2 t/s gen
Quant
IQ3_XXS (GGUF)
System RAM
128 GB

Use cases

agentic

Summary

User reports a Qwen3.8-Flash-Next MTP draft head for GGUFs on llama.cpp, measured on an M3 Max 128GB with Metal. Setup is llama.cpp with an IQ3_XXS trunk and an unsloth shared Q8_0 MTP head, one session. Decode is 31.7-35.2 tok/s with MTP against 23.8-27.4 without, draft acceptance 84-99% at 2 drafts. Five concurrent agent sessions cost 8-20% throughput at 2 drafts, while 1 draft at 89% acceptance is neutral. The work is blocked on unmerged llama.cpp PR #28243.