llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 3× NVIDIA RTX 3090 · vLLM · 1,000,000 ctx

Tone: positiveNVIDIA hardware
Sep 16, 2026
Throughput
80.0 t/s gen · 3701.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Use cases

long-context

Summary

User reports Qwen3.8-Flash-Next at ~80 t/s on 3x RTX 3090 at 1M context, dropping to ~60 t/s once QSA reaches its 2048-token budget. Setup is vLLM with most of the KV cache offloaded to system RAM, no KV cache quantization. Prefill at 248k reaches 3,701 t/s, and throughput is ~150 t/s at 4 concurrent requests. Decode speed stays flat as total context grows because only 12 of 48 layers hold a KV cache and QSA bounds the per-step read to 48 MiB per token.