llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4080 Super · QwFNfer · 160,000 ctx

NVIDIA hardware
Oct 2, 2026
Throughput
12.9 t/s gen · 355.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
KV cache
q8_0
System RAM
30 GB
VRAM reported
16 GB

Summary

User reports Qwen3.8-Flash-Next at 12.9 tok/s decode and 355 tok/s prefill at 155K context on an RTX 4080 Super 16 GB. Setup is the QwFNfer engine with UD-Q4_K_XL weights and q8_0 KV cache; the dense core and a VRAM expert tier stay on the card while the remaining experts are read from 30 GB of RAM and NVMe, with attention state partly in pinned RAM. Short chat decode is 13.2-15.7 tok/s for Q4 and 19.8-21.2 tok/s for Q3; at 155K context Q3 gives 348 tok/s prefill and 17.7 tok/s decode. A speculative block predicts the next layer's experts at 95-96% accuracy, with no token-level speculative decoding.