llamaperf
Oct 11, 2026
Throughput
118.6 t/s gen
Quant
NVFP4 / MXFP8 mixed (safetensors)
KV cache
FP8
MTP (Multi-Token Prediction)
on
System RAM
256 GB

Use cases

coding

Summary

User reports Qwen3.8 at 118.6 t/s on two NVIDIA DGX Spark (GB10 Grace Blackwell) machines at TP=2. Setup is vLLM with NVFP4 experts and MXFP8 dense/attention paths, FP8 KV cache, probabilistic MTP with 4 draft tokens, and a configured max_model_len of 262144. The figure is the mean of 3 boot-level means of per-pass median request decode rates over 36 coding prompts run one at a time, with a between-boot sample SD of 0.353 t/s; the workload is favorable to MTP and is not a general conversational speed estimate.