llamaperf

GLM-5.3 320B (18B active) Flash

on NVIDIA RTX 3090 · Strata · 32,768 ctx

NVIDIA hardware
Oct 9, 2026
Throughput
19.1 t/s gen · 41.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
GSQ-RCO 3.5-bit (GGUF)
System RAM
177 GB
VRAM reported
24 GB

Summary

User reports GLM-5.3-Flash at 19.1 t/s decode and 41 t/s prefill on an RTX 3090 with a 47-token prompt. Setup is Strata with GSQ-RCO 3.5-bit GGUF at 32,768 tokens of context, most experts in RAM (121 GiB of 177 GiB system RAM used). Prefill rises with prompt length to 974 t/s at 26,641 tokens; decode stays 18.7-19.0 t/s across contexts. On 9 GPUs (8x RTX 5060 Ti 16 GB plus the 3090) decode is about 28 t/s and 8K prompts run 1150-1210 t/s. Calibration kept the engine's own settings at 21.6 t/s.