llamaperf

GLM-5.3 320B (18B active) Flash

on 4× NVIDIA DGX Spark · vLLM

Tone: positiveNVIDIA hardware
Oct 3, 2026
Throughput
69.3 t/s gen
Quant
NVFP4 (NVFP4)
KV cache
FP8 e4m3

Use cases

coding

Summary

User reports GLM-5.3-Flash at 69.3 tok/s single-stream coding peak on four NVIDIA DGX Spark (GB10) nodes at tensor-parallel 4. Setup is vLLM with NVFP4 weights and FP8 e4m3 KV cache, DFlash2 block-diffusion speculative decoding with a 5-layer Qwen3 drafter proposing 7 tokens per step; ~45 GiB of weights per node. Mixed-prompt sustained decode is 36.7 tok/s at c1 (42.2 tok/s at temp=0), 56.1 tok/s aggregate at c2 and 84.7 tok/s at c4; prefill holds ~1,700 tok/s from 4K to 128K context. A v3 config with pinned KV cache and chunked prefill regressed 25% to 31.5 tok/s.