llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA DGX Spark · vLLM · 262,144 ctx

Tone: positiveNVIDIA hardware
Sep 28, 2026
Throughput
60-70 t/s gen
Quant
W4A16 AutoRound (AutoRound)
System RAM
128 GB

Use cases

codingagenticlong-context

Summary

User reports Qwen3.8 Flash at 60-70 (peak 74.1) tok/s peak single-stream decode on one DGX Spark (GB10), with 60 to 70 tok/s on normal requests and 212.2 tok/s aggregate across 8 streams. Setup is vLLM with W4A16 AutoRound for the MoE experts, FP8 side layers and INT8 lm_head, full 262K context, using the model's own MTP draft head with a draft vocab cut from 248K to 65K ids. Cold prefill reaches 4,016 tok/s at 16K, 2,426 tok/s at 64K and 2,213 tok/s at 128K, versus 1,171 / 1,071 / 1,065 tok/s for the original recipe. Cached coding prompts start replying in about 0.57 s. Quality scored 93.1% and 93.3% on a fixed 492-question suite over two seeds. The model takes about 71 GiB with a 16 GB KV pool; CJK drafting is slower due to the English/code draft vocab.