GLM-5.3 320B (18B active) Flash
on 2× NVIDIA RTX Pro 6000 Blackwell · vLLM · 1,048,576 ctx
Oct 3, 2026
Use cases
long-contextvisionagentictool-use
Summary
User reports GLM-5.3 Flash at 179.50 tok/s single-stream decode on two RTX PRO 6000 Blackwell 96GB GPUs.
Setup is vLLM with EXL3-K3 quant and FP8 KV cache, TP2 + EP2 + DCP2, DFlash2 speculative decoding, 1,048,576-token context ceiling.
Aggregate decode reaches 1,045.47 tok/s at concurrency 16; prefill is 4,107-4,258 tok/s across 8K-128K prompts. Exact one-million-token six-needle retrieval passed 6/6 in 284.859 seconds.