Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX 4080 Super · QwFNfer · 160,000 ctx
Summary
User reports Qwen3.8-Flash-Next at 12.9 tok/s decode and 355 tok/s prefill at 155K context on an RTX 4080 Super 16 GB.
Setup is the QwFNfer engine with UD-Q4_K_XL weights and q8_0 KV cache; the dense core and a VRAM expert tier stay on the card while the remaining experts are read from 30 GB of RAM and NVMe, with attention state partly in pinned RAM.
Short chat decode is 13.2-15.7 tok/s for Q4 and 19.8-21.2 tok/s for Q3; at 155K context Q3 gives 348 tok/s prefill and 17.7 tok/s decode. A speculative block predicts the next layer's experts at 95-96% accuracy, with no token-level speculative decoding.