llamaperf

GLM-5.3 320B (18B active) Flash

on 2× NVIDIA RTX Pro 6000 Blackwell · SGLang · 524,288 ctx

Tone: positiveNVIDIA hardware
Oct 12, 2026
Throughput
164.8 t/s gen
Quant
W4A16-NVFP4-K32-Experts-FP8-WO (safetensors)
KV cache
FP8 E4M3
VRAM reported
192 GB

Use cases

agenticcodingtool-uselong-context

Summary

User reports GLM-5.3-Flash at 164.8 tok/s single-stream decode (388.2 tok/s aggregate at four concurrent requests) on two RTX PRO 6000 Blackwell GPUs. Setup is SGLang v0.4.3 SM120 with the ormandj W4A16-NVFP4-K32-Experts-FP8-WO checkpoint, FP8 E4M3 KV cache, 524,288-token context, adaptive EAGLE 5/1/6 speculative decoding and tensor parallel size 2. Cold prompt processing reaches 5,174 tok/s at 2K and 5,961 tok/s at 32K. Four simultaneous requests is the tuned operating point; at eight, aggregate falls 7.7% and median time to first token rises to 1,428 ms. The 4- and 8-GPU capacity figures are arithmetic projections, not measurements.