GLM-5.3 320B (18B active) Flash
on 2× NVIDIA RTX 3090 · llama.cpp · 65,536 ctx
Oct 9, 2026
Summary
User reports GLM-5.3-Flash at 19.17 tok/s decode on 2x RTX 3090.
Setup is llama.cpp with Q2_K Medium GGUF and F16 KV cache, 65536 context, layer split across two cards with static expert offload to CPU and an external DFlash2 Q4_K_M drafter (SPEC_N=2).
Engine-side deep-prefill mean 117.78 tok/s; 10K prefill mean 118.99 tok/s. DFlash fired on 16/16 requests with mean acceptance 0.681. Both cards power-capped at 210 W. Soak-continuous was skipped.