GLM-5.2

reported speed:

7.3 tokens/s generation · 135.0 tokens/s prompt processing

quant:

UD-IQ2_M

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

4x RTX 3090 (1 Ti + 3 FE), 96GB total VRAM, but only ~83GB used. Model: GLM-5.2 UD-IQ2_M, 223GB on disk, 744B total / 40B active. Decode at ~7.3 tok/s with expert offload to CPU. Prefill flat at 135 tok/s. A/B tests: IQ1_M vs IQ2_M identical (7.30 vs 7.29 tok/s); 6 vs 12 CPU threads gave +22% (5.83 vs 7.14 tok/s). Decode bound by CPU compute, not bandwidth. KV cache ~6GB at 128K, scales linearly. Thinking off gives ~5-6x wall-clock speedup. Setup: llama.cpp master, glm-dsa arch, explicit expert placement via -ot to avoid OOM.

reported speed:

14.8 tokens/s generation · 512.0 tokens/s prompt processing

quant:

NVFP4

kv:

fp8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

4x DGX Spark setup with GLM-5.2 NVFP4, 128K context, TP4/PP1/DCP4/MTP1, fp8 KV cache, ~15 t/s decode short, ~13 t/s at long context, ~512 t/s prefill.

reported speed:

6.0 tokens/s generation

quant:

Q1_S

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

GLM-5.2 Q1_S outperformed Qwen 3.6 27B Q8 in code generation quality despite lower quant. GLM ran at ~6 tps (low context) down to 3 tps (100k context). Qwen ran at ~60 tps. Both models run under pi harness. GLM FP via OpenRouter also tested but Q1_S held up well.

User built a system with 5x RTX Pro 6000 and 1x RTX 5090, running GLM 5.2. Reports 98-99% task completion but notes high heat and cost. No specific t/s numbers given.

reported speed:

24.0 tokens/s generation · 475.0 tokens/s prompt processing

quant:

NVFP4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

MTP3 default, MTP4 peak. Bug fix: draft parallel config missing DCP copy. Prefill ~475 tps, bs=3 decode ~48 tps.