GLM-5.3 320B (18B active) Flash
on NVIDIA RTX 5090 · Project Maya · 131,072 ctx
Summary
User reports 26.3 t/s generation and 2165 t/s prompt processing.
Source evidence (verbatim):
Source: https://dev.to/seppegadeyne/run-glm-53-flash-321b-locally-with-project-maya-and-exl3-on-one-rtx-5090-24pj
# Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090
Hardware: one RTX 5090 32 GB (also driving the desktop), Ryzen 9 9950X3D, 128 GB DDR5 at 5,600 MT/s, Arch Linux, NVIDIA driver 615.78, CUDA 13.4. Engine: Project Maya 1.1.11 (a Strata fork) compiling llama.cpp at a pinned commit. Model: GLM-5.3-Flash 321B MoE with about 18B active, turboderp EXL3 3.05 bpw (125.3 GB), with the expert cache and KV streaming offloading to system RAM. Benchmark: streaming client, 8 runs, greedy decoding, 512 output tokens, median reported. At 128K context with the default reserve: decode 26.3 tokens/s (30.4 on Dutch prose, 21.9 on code) and prefill 2,165 tokens/s at 126K. At 262K context: decode 24.1 tokens/s and prefill 2,039 tokens/s at 254K.