llamaperf

GLM-5.3 320B (18B active) Flash

on NVIDIA RTX 5090 · strata-glm · 65,536 ctx

Tone: mixedNVIDIA hardware
Oct 8, 2026
Throughput
15-18 t/s gen · 1000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (NVFP4)
System RAM
128 GB
VRAM reported
32 GB

Summary

User reports GLM-5.3-Flash NVFP4 on one RTX 5090 at ~15-18 tok/s decode and ~1000 tok/s prefill on a 64K-token prompt. Setup is strata-glm, a Strata fork, with the NVFP4 checkpoint on one RTX 5090, experts streamed from 128 GB DDR5 RAM and two NVMe drives. Decode on a ~1-1.4K chat prompt was 18.0 tok/s on Cyrillic chat and 14.8 on code by default, 16.5 and 12.9 in exact mode, and 15.3 in exact mode after the 64K prompt. llama.cpp on the same box managed 2.4-2.9 tok/s decode and 29 tok/s prefill. The project is archived.