GLM-5.3 320B (18B active) Flash
2× NVIDIA RTX 4060 Ti 16GB · Strata · 131,072 ctx
- generation:
- 13.0 tokens/s
- quant:
- UD-IQ3_XXS (GGUF)
User reports GLM-5.3-Flash at a mean 13.0 tokens/s decode on 2x RTX 4060 Ti 16GB, with prefill of 165-166 tokens/s at 2048 and 385-386 tokens/s at 8192. Setup is Strata v1.1.42 built from source with the unsloth UD-IQ3_XXS GGUF (4 shards) at 131072 context, expert cache auto and KV streaming on, model on a SATA SSD. With STRATA_GLM_DEFER_CPU=1 decode was 13.4 and 13.7 tokens/s; with =2 (a separate PR) it was 14.0 tokens/s on both starts.