llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA DGX Spark · vLLM · 262,144 ctx

Tone: mixedNVIDIA hardware
Oct 18, 2026
Throughput
45.8 t/s gen · 1717.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
KV cache
FP8
System RAM
121 GB

Use cases

tool-uselong-context

Summary

User benchmarks Qwen3.8-Flash-Next served four ways on one DGX Spark (GB10, 121 GB unified memory), reporting 45.8 t/s decode and 1,717 t/s prefill for their own fast recipe. Setup is vLLM with NVIDIA's NVFP4 FP4-Local checkpoint, FP8 KV cache and MTP 3, at a 262,144-token window with thinking off and temperature 0. The other recipes reach 54.3 t/s (hibrid48) and 51.6 t/s (TensorFold) decode; quality mode drops to 44.4 t/s and 1,362 t/s prefill. Under concurrent load hibrid48 delivers 82 t/s with 16 requests of 8k against 51 for the user's recipe. TensorFold returns HTTP 400 on response_format and scores 6/12 on structured output versus 10/12 for the vLLM recipes.