Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 5080 · llama.cpp · 262,144 ctx
Sep 18, 2026
Summary
User reports Qwen3.8-Flash-Next at around 100 t/s prefill and 18-20 t/s decode on 2x RTX 5080 with 128 GB DDR5.
Setup is llama.cpp with the unsloth UD-Q5_K_XL GGUF, a Q8_0 KV cache, 262,144 context, MTP speculative decoding with a Q4_K_M draft model, and ncmoe42 offloading 42 MoE layers to CPU.
User asks whether switching to vLLM or SGLang would give better performance, noting the scarcity of NVFP4/FP8 GGUF files.