Qwen3.8 27B
on NVIDIA RTX 3090 · llama.cpp · 8,192 ctx
Oct 12, 2026
Summary
User reports Qwen3.8-27B at 66.39 tok/s on one RTX 3090, versus a 41.55 tok/s non-speculative baseline, a +59.8% within-host effect.
Setup is llama.cpp with Q4_K_XL GGUF weights, q8_0 KV cache, 8192-token context, flash attention on, single stream, batch size 1, using the built-in MTP path at --spec-draft-n-max 2.
The faster path was not bit-exact with serial greedy decoding: 92-100% of requests diverged by a 1,600-token cap, and semantic quality was not measured. Deeper drafting was slower (MTP n-max 3 at 63.29 tok/s, DFlash2 n-max 4 at 63.13, MTP n-max 5 at 54.89, DFlash2 n-max 7 at 50.95). Board telemetry estimated a 37% decode energy reduction for a fixed 400-token workload.