llamaperf

Qwen3.8 27B Heretic

on NVIDIA RTX 4080 · llama.cpp · 32,768 ctx

Tone: positiveNVIDIA hardware
Oct 11, 2026
Throughput
55.0 t/s gen
Quant
UD-IQ4_XS (GGUF)
VRAM reported
16 GB

Summary

User reports Qwen3.8-27B Heretic (UD-IQ4_XS) at 55 tok/s on code and 50 tok/s on prose on an RTX 4080 16GB with MTP 2 draft tokens. Setup is llama.cpp b11457 with a 15K-token prompt, 32K context, MTP 2 drafts, mean of 3 fixed seeds. Without MTP the same config gives 29 tok/s on both code and prose. User notes 2 draft tokens beat 3 (68 vs 56 tok/s on code, 55 vs 36 on prose at 40K), 48K vs 40K produced identical tokens but was 22% slower, and VRAM spill from the desktop dropped code decode to 68, 51 or 42 tok/s depending on 1.2, 1.4 or 1.9 GB of shared VRAM. MTP head precision barely matters (q6_K vs IQ3_S: 83% vs 83% acceptance on code, 58% vs 54% on prose). The 12GB and 24GB context numbers are computed from llama.cpp buffers, not measured.