llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 4× NVIDIA RTX 3090 · vLLM · 65,536 ctx

Tone: positiveNVIDIA hardware
Sep 27, 2026
Throughput
116.6 t/s gen
Quant
W4A16 (W4A16)

Use cases

codinglong-context

Summary

User reports Qwen3.8-Flash-Next 180B W4A16 at 116.6 t/s decode on four RTX 3090s capped at 220 W each. Setup is vLLM built from a pinned PR head with five patches, MTP speculative decoding with k=4, 65,536 context, and 76,046 KV tokens. Decode holds 115.1 t/s at 32k and 72.75 t/s at 128k; prefill reads 3,187 t/s at 32k and 2,835 t/s at 128k. The c8 aggregate is 180.7-230.3 t/s on the bf16 profile. Peak short-prompt decode reaches 198.5 t/s on code. Spread is plus or minus 15% run to run.