Qwen3.8 125B (6B active) Flash-Next
on DGX Station (GB300) · SGLang
Summary
User reports Qwen3.8-Flash-Next at 221 t/s single-stream on a DGX Station (ASUS ET900N G3, GB300, ~250 GB VRAM, 72 Grace cores, 750 GB RAM) with an RTX PRO 6000 also in the case.
Setup is SGLang with NVFP4 weights; the same model on vLLM gave 219 t/s single-stream.
Aggregate throughput was 1537 t/s at 16 parallel requests and 3002 t/s at 64 on SGLang, versus ~1370 and ~2700 on vLLM. DeepSeek V4.1 Flash on a vLLM nightly reached 59 t/s single and 246 t/s at 16 parallel, taking 19 minutes to load. The GB300 drew around 785 W at 50 °C. The user asks what to benchmark during a month of access.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3.8 125B · 6B active at 4-bit by hardware
- 129.2 t/s · one run
- 105.5 t/s · 6 runs
- 57.2 t/s · 2 runs
- 55.0 t/s · one run
- 36.7 t/s · one run
- NVIDIA DGX Spark128 GB29.8 t/s · 2 runs
Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.