llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 3090 · vLLM · 126,000 ctx

NVIDIA hardware
Sep 15, 2026
Throughput
40.0 t/s gen · 1104.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ4_XS (GGUF)

Summary

User reports Qwen3.8 Flash-Next at ~40 t/s decode on dual RTX 3090 (sm_86) with the IQ4_XS GGUF, stable from 1k to 126k context. Setup is a vLLM 0.29.0 GGUF-capable fork with patches and env toggles; 32k fresh prefill measured at ~1104 t/s. Q4_K_XL is kept experimental: ~32.3 t/s decode at 1k, ~854 t/s prefill at 8k, and unstable at long context. An older ~6665 t/s prefill figure was found non-reproducible.