GLM-5.3 320B (18B active) Flash
on NVIDIA RTX Pro 6000 Blackwell · llama.cpp
Oct 9, 2026
Summary
User reports GLM-5.3-Flash at 69.89 t/s on an RTX PRO 6000 Blackwell 96 GB.
Setup is llama.cpp with an IQ2_XXS GGUF and Q8_0 KV cache, fully resident on the card.
Decode falls to 52.74 t/s at 8192 context, 31.70 at 32768 and 12.24 at 131072. The same file on a single RTX 3090 24 GB with 64 GB DDR5 and -ncmoe 40 gives 11.55 t/s decode and 36.69 t/s prefill. The IQ3_XXS build is not benchmarked.