llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 5080 · llama.cpp · 262,144 ctx

Tone: mixedNVIDIA hardware
Sep 18, 2026
Throughput
18-20 t/s gen · 100.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q5_K_XL (GGUF)
KV cache
q8_0
System RAM
128 GB

Summary

User reports Qwen3.8-Flash-Next at around 100 t/s prefill and 18-20 t/s decode on 2x RTX 5080 with 128 GB DDR5. Setup is llama.cpp with the unsloth UD-Q5_K_XL GGUF, a Q8_0 KV cache, 262,144 context, MTP speculative decoding with a Q4_K_M draft model, and ncmoe42 offloading 42 MoE layers to CPU. User asks whether switching to vLLM or SGLang would give better performance, noting the scarcity of NVFP4/FP8 GGUF files.