Use cases
rp
Summary
User reports GLM-5.3-Flash at about 11 tok/s on an RX 9070 XT 16GB, with long prompts read at about 280 tok/s on Linux.
Setup is the user's own engine with IQ4_XS GGUF, experts streamed from NVMe drives with hot experts kept on the GPU and in RAM, on a Core Ultra 7 265KF with 32 GB DDR5 and two NVMe drives.
Windows gives about 10 tok/s and about 250 tok/s prompt reading. Testers measured about 11 tok/s on an Intel Arc Pro B70 32GB and 5.6 tok/s on an RTX 3090 plus RTX 3060; the 12-13 and 6.5-7 tok/s figures are expectations from logs, not measurements. The model needs about 157 GB on disk and 32 GB of RAM.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
GLM-5.3 320B · 18B active at 4-bit by hardware
- 37.4 t/s · one run
Bars are medians on one scale, 0 to 37.4 tokens/s. Press ▶ to watch text arrive at a row's speed.
Models most run on the AMD RX 9070 XT 16GB
- Qwen3 8B4-bit98.8 t/s · one run
- Qwen3.5 9B4-bit7.9 t/s · one run
- Qwen3.5 9B6-bit62.0 t/s · one run
Bars are medians on one scale, 0 to 98.8 tokens/s. Press ▶ to watch text arrive at a row's speed.
Everything reported on the AMD RX 9070 XT 16GB →Every GLM-5.3 report →Estimate GLM-5.3 on this card →