GLM-5.3 320B (18B active) Flash
on 2× NVIDIA RTX 3090 · llama.cpp · 32,768 ctx
Oct 9, 2026
Summary
User reports GLM-5.3-Flash at 21.5 t/s decode on 2x RTX 3090 with a VRAM-filling MoE expert cache, up from 13.8 t/s without it.
Setup is a neurall/llama.cpp fork with a GSQ-RCO 3.0-bit GGUF (Q4_K attention), 32k context, experts in RAM with a GPU expert cache, on a Ryzen 7 3700X with 125 GB DDR4.
Long 12k-token code prompt: 181 t/s prompt processing and 15.7 t/s decode with the cache, versus 217 t/s and 12.3 t/s without. Perplexity 3.5871 on wikitext-2, 0.95% above the original quant's 3.5534.