llamaperf

GLM-5.3 320B (18B active) Flash

on 2× NVIDIA RTX 5090 · llama.cpp · 204,800 ctx

NVIDIA hardware
Oct 6, 2026
Throughput
22.0 t/s gen · 421.1 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ4_XS (GGUF)
KV cache
f16
System RAM
251 GB

Use cases

codingagenticlong-contexttool-usevision

Summary

User reports GLM-5.3-Flash at 21.96 t/s decode (narrative) and 26.10 t/s (code) on 2x RTX 5090. Setup is llama.cpp with IQ4_XS GGUF weights, f16 KV cache, 204800-token context, DFlash2 Q4_K_M draft model with speculative decoding (n=3), and MoE expert tensors offloaded to CPU. Prefill measured 421.07 t/s at ~10K prompt tokens and 448.20 t/s at ~87K. Soak test passed with p50 decode 20.70 t/s and 100% TPS retention over 25 turns. Expert cache was requested but not allocated (CACHE_DISABLED), so the run measured a half-cached path. Stress test failed the long-context recall ladder.