Qwen3.8 125B (6B active) Flash-Next
2× NVIDIA T4 16GB · Strata · 196,608 ctx
- generation:
- 38.7 tokens/s
- prompt processing (prefill):
- 1160.0 tokens/s
- quant:
- UD-Q4_K_XL (GGUF)
User reports Qwen3.8-Flash-Next at 38.7 t/s decode and ~1,160 tok/s prefill on a 189,104-token prompt with a 196,608-token live context, on 2x Nvidia Tesla T4 16GB. Setup is Strata with UD-Q4_K_XL GGUF, expert cache in VRAM and spill to host RAM, --kv-resident 32768. The same user runs Qwen3.6-35B-A3B UD-Q4_K_M in llama.cpp at 55 t/s decode (up to 70 depending on context size and reasoning effort) with ~1k t/s prefill at 237,568 context, entirely in VRAM. Peak GPU power during the context test was 149W, with the server averaging ~320W.