llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4080 · llama.cpp · 131,072 ctx

NVIDIA hardware
Sep 24, 2026
Throughput
16.5 t/s gen · 350.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
KV cache
Q8
System RAM
64 GB
VRAM reported
16 GB

Summary

User reports Qwen3.8 Flash-Next at 16.5 t/s generation and 350 t/s prompt processing on an RTX 4080 16GB with 64GB system RAM, at 131k context. Setup is llama.cpp built from a pr-mtp-fix branch with the Unsloth MTP head as the draft model, AtomicChat AD-4.27bpw Q4_K_M target weights, and a Q8 KV cache. The --spec-draft-cpu-moe flag keeps draft experts in RAM so the target's hot experts fit in the 16GB of VRAM. An IQ4_XS quant was also tested and reached about 10 t/s.