llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 3090 · 262,144 ctx

NVIDIA hardware
Sep 22, 2026
Throughput
135.2 t/s gen · 1402.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
W4A16 (AutoRound)
System RAM
128 GB
VRAM reported
24 GB

Summary

User reports Qwen3.8-Flash-Next at 1,402 prompt tok/s and 135.2 output tok/s on 2x RTX 3090 with 128 GB system memory. Setup uses the Intel AutoRound W4A16 target with FP8 PLE/ngram tensors. Prefill and decode use different test shapes and did not come from the same request. A 262,016-token prompt with 128 output tokens prefilled at 1,275.6 tok/s and decoded at 54.5 tok/s. Warm decode over 4,096 output tokens ranged 127.1-134.0 tok/s. Maximum tested sequence was 262,144 tokens.