llamaperf

GLM-5.3 320B (18B active) Flash

on NVIDIA RTX 3090 · glm53-flash-offload

NVIDIA hardware
Oct 9, 2026
Throughput
17.8 t/s gen
VRAM reported
16 GB

Summary

User reports GLM-5.3-Flash at 17.8 tok/s decode and 996 tok/s prefill on an RTX 3090 with the 55 GB NVMe offload launch. Setup is the glm53-flash-offload v4.4-nvme image, with the model's weights streamed from NVMe (55 GB) plus 16 GB resident; the 16 GB launch measured 13.3 tok/s decode and 986 tok/s prefill and failed the 15 tok/s speed gate. Previous runs were 17.3 tok/s for 55 GB and 13.1 tok/s for 16 GB. The 55 GB launch passed all gates and was accepted; the 16 GB launch was reported. Host settings outside the launch hash were --gpus device=0 and --cpuset-cpus 2-39 with no read cap.