llamaperf

GLM-5.2 744B (40B active)

on NVIDIA RTX 3090 · llama.cpp · 131,072 ctx

Tone: positiveNVIDIA hardware
Sep 7, 2026
Throughput
7.3 t/s gen · 135.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-IQ2_M
KV cache
q8_0
System RAM
192 GB
VRAM reported
24 GB

Summary

User reports GLM-5.2 UD-IQ2_M at ~7.3 tok/s decode and 135 tok/s prefill on 4x RTX 3090 (1 Ti + 3 FE), 96GB total VRAM with only ~83GB used. Setup is llama.cpp master with the glm-dsa arch, a 223GB on-disk model at 744B total / 40B active, expert offload to CPU, explicit expert placement via -ot to avoid OOM, and a ~6GB KV cache at 128K that scales linearly. Decode is bound by CPU compute, not bandwidth. IQ1_M and IQ2_M are identical in A/B tests (7.30 vs 7.29 tok/s), and 6 vs 12 CPU threads gave +22% (5.83 vs 7.14 tok/s). Thinking off gives ~5-6x wall-clock speedup.