llamaperf
Nov 18, 2026
Throughput
11.0 t/s gen · 280.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
IQ4_XS (GGUF)
System RAM
32 GB
VRAM reported
16 GB

Use cases

rp

Summary

User reports GLM-5.3-Flash at about 11 tok/s on an RX 9070 XT 16GB, with long prompts read at about 280 tok/s on Linux. Setup is the user's own engine with IQ4_XS GGUF, experts streamed from NVMe drives with hot experts kept on the GPU and in RAM, on a Core Ultra 7 265KF with 32 GB DDR5 and two NVMe drives. Windows gives about 10 tok/s and about 250 tok/s prompt reading. Testers measured about 11 tok/s on an Intel Arc Pro B70 32GB and 5.6 tok/s on an RTX 3090 plus RTX 3060; the 12-13 and 6.5-7 tok/s figures are expectations from logs, not measurements. The model needs about 157 GB on disk and 32 GB of RAM.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

GLM-5.3 320B · 18B active at 4-bit by hardware

Bars are medians on one scale, 0 to 37.4 tokens/s. Press ▶ to watch text arrive at a row's speed.

Models most run on the AMD RX 9070 XT 16GB

Bars are medians on one scale, 0 to 98.8 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the AMD RX 9070 XT 16GB →Every GLM-5.3 report →Estimate GLM-5.3 on this card →