GLM-5.3 320B (18B active) Flash
on NVIDIA RTX 3090 · ExLlamaV3 · 128,000 ctx
Oct 9, 2026
Summary
User reports GLM-5.3-Flash at 17.3 tok/s decode on an RTX 3090.
Setup is ExLlamaV3 with a 3.05bpw EXL3 quant in NVMe mode (55 GB), 128k context, single card. The context gate read 93,437 prompt tokens at 844 tok/s prefill.
An NVMe read cap of 8 GB/s cost about 8% of C1 decode and 26% of prefill; uncapped same-session figures were C1 18.23 and C4 19.59 tok/s. Capped standard table: 8k C1 16.61, C2 14.91, C4 13.10; 32k C1 14.43, C2 15.41.