GLM-5.3 320B (18B active) Flash
on NVIDIA RTX 5090 · strata-glm · 65,536 ctx
Summary
User reports GLM-5.3-Flash NVFP4 on one RTX 5090 at ~15-18 tok/s decode and ~1000 tok/s prefill on a 64K-token prompt.
Setup is strata-glm, a Strata fork, with the NVFP4 checkpoint on one RTX 5090, experts streamed from 128 GB DDR5 RAM and two NVMe drives.
Decode on a ~1-1.4K chat prompt was 18.0 tok/s on Cyrillic chat and 14.8 on code by default, 16.5 and 12.9 in exact mode, and 15.3 in exact mode after the 64K prompt. llama.cpp on the same box managed 2.4-2.9 tok/s decode and 29 tok/s prefill. The project is archived.