llamaperf

GLM-5.3 320B (18B active) Flash

on 2× NVIDIA RTX Pro 6000 Blackwell · vLLM · 1,048,576 ctx

NVIDIA hardware
Oct 3, 2026
Throughput
179.5 t/s gen
Quant
EXL3-K3 (EXL3)
KV cache
FP8
VRAM reported
96 GB

Use cases

long-contextvisionagentictool-use

Summary

User reports GLM-5.3 Flash at 179.50 tok/s single-stream decode on two RTX PRO 6000 Blackwell 96GB GPUs. Setup is vLLM with EXL3-K3 quant and FP8 KV cache, TP2 + EP2 + DCP2, DFlash2 speculative decoding, 1,048,576-token context ceiling. Aggregate decode reaches 1,045.47 tok/s at concurrency 16; prefill is 4,107-4,258 tok/s across 8K-128K prompts. Exact one-million-token six-needle retrieval passed 6/6 in 284.859 seconds.