GLM-5.3 reports

generation:

164.8 tokens/s

quant:

W4A16-NVFP4-K32-Experts-FP8-WO (safetensors)

kv:

FP8 E4M3

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 164.8 tok/s single-stream decode (388.2 tok/s aggregate at four concurrent requests) on two RTX PRO 6000 Blackwell GPUs. Setup is SGLang v0.4.3 SM120 with the ormandj W4A16-NVFP4-K32-Experts-FP8-WO checkpoint, FP8 E4M3 KV cache, 524,288-token context, adaptive EAGLE 5/1/6 speculative decoding and tensor parallel size 2. Cold prompt processing reaches 5,174 tok/s at 2K and 5,961 tok/s at 32K. Four simultaneous requests is the tuned operating point; at eight, aggregate falls 7.7% and median time to first token rises to 1,428 ms. The 4- and 8-GPU capacity figures are arithmetic projections, not measurements.

generation:

8-9 tokens/s

quant:

W2/W3/W4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at about 8-9 tok/s on dual Ascend 310P 96GB cards. Setup is vllm-ascend with W2/W3/W4 quantization, 311,040 tokens per request in 640-token prefill chunks, up to 4 concurrent requests. Cold prefill is lengthy; subsequent prompts run quicker. User also runs Qwen3.8-Flash-Next on the same hardware with YaRN support for 1M context.

generation:

30.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3 Flash at 30 tok/s with Project Maya, up from about 10 tok/s previously. Setup is Project Maya, an engine built for this model, with experts kept on GPU, RAM and NVMe SSD. The user finds GLM-5.3 Flash more enjoyable than Qwen 3.8 Flash-Next so far and is still testing.

generation:

48.8 tokens/s

quant:

4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash Abliterated at 48.8 tok/s decode on a Mac Studio M3 Ultra 256 GB. Setup is mlx-serve 26.10.1 with MLX 4-bit weights and 131072 context, MTP on with draft rerank, resident size about 172–181 GB. Real sessions at 50k–85k tokens decode about 45–63 tok/s with prefill 150–300 tok/s on cache hits. A short probe at 8192 context measured 49.9 tok/s. TensorFold 1.0.5 on the same Mac left the MTP head on disk and one short reply measured 15.7 tok/s.

generation:

18.3 tokens/s

prompt processing (prefill):

320.0 tokens/s

quant:

Maya-S-v2-IQ2_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3 Flash (Maya-S-v2 IQ2_XXS) at 18.3 t/s generation and 320 t/s prompt processing on an AMD Strix Halo 128GB. Setup is Project Maya 1.0.30 on HIP with the Maya-S-v2-IQ2_XXS mixed quant, one request, 262K configured context, the whole model in unified memory; whether MTP was on is not stated. Source: https://github.com/mw00/project-maya/issues/102 (LIN-LAB-AI, 2026-10-11; third-party measurement, not our run). Headline metrics are separate built-in --bench tests: 320 tok/s prefill for 8,192 input tokens; generation 18.3 tok/s reported mean, three 256-token answers (18.3/18.4/18.3), greedy, thinking off. Short/warm decode is not generation following that 8K prefill. Also: 267 tok/s prefill for 2,048 input tokens. Prefill repetition count unspecified. Configured context 262,144, not headline prompt length. Hardware/runtime: one Radeon 8060S (gfx1151), Ryzen AI Max+ 395, 128GB unified; 96GiB BIOS carve-out. Windows reports 31.65GiB host RAM; HIP reports 107.87GB GPU memory. These are overlapping unified-memory views, not additive capacities. Native Windows 11 build 26100; HIP SDK 7.2/clang21; Maya 1.0.30 c189f7a; --backend hip --gpu 0; NVMe model storage; --prefill auto, PREFILL_SUB=1024, PREFILL_MB=6144. Separate cold technical-prose sweep asserts reused=0 and uses engine prompt_ms/decode_ms: at 8,632–9,070 input tokens median 315.1 pp / 15.90 tg; at 261,748–261,771 input median 274.5 pp / 14.20 tg. Answers usually ~65 tokens, so not comparable to 256-token headline. No downloadable per-request sweep logs linked. KV precision, Flash Attention, MTP status, clocks/power and exact device manufacturer unspecified. Verified model identity: peasantsmith/GLM-5.3-Flash-Maya-GGUF, folder Maya-S-v2-IQ2_XXS, three GLM-5.3-Flash-Maya-S-v2-IQ2_XXS-0000{1,2,3}-of-00003.gguf shards. Source reports SHA256 verification against its launcher. Tested-version manifest: https://github.com/mw00/project-maya/blob/v1.0.30/maya.py . It accepts two first-shard headers and uses revision main; exact tested Hugging Face commit is unknown. Quant is mixed: IQ2_XXS gate/up experts, IQ2_S/IQ3_XXS down, Q6_K attention, including an MTP block (presence does not establish use). Model card/file verification: https://huggingface.co/peasantsmith/GLM-5.3-Flash-Maya-GGUF/tree/main/Maya-S-v2-IQ2_XXS and https://huggingface.co/peasantsmith/GLM-5.3-Flash-Maya-GGUF . Card identifies 321B total/~18B active and 96.5GB text weights. Source 90.90GiB on-disk includes vision files; not peak RAM. Vision was untested. No quality benchmark reported.

generation:

61.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash (320B) at ~61 tok/s generation on an M5 Ultra Mac Studio with 256GB unified memory. Setup is llama.cpp with GLM's built-in MTP head (probabilistic drafting, 3 draft tokens); without MTP generation is 39.7 tok/s. The full 262K context fits with ~15 GiB to spare. Prompt reading is ~765 tok/s at 4K tokens and ~630 tok/s at 59K; a 29K-token prompt takes 45 s. Long-context recall was correct at 58.8K tokens. Images and MTP cannot be enabled simultaneously, and deep context on Metal needs an unmerged fix.

generation:

19.1 tokens/s

prompt processing (prefill):

41.0 tokens/s

quant:

GSQ-RCO 3.5-bit (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 19.1 t/s decode and 41 t/s prefill on an RTX 3090 with a 47-token prompt. Setup is Strata with GSQ-RCO 3.5-bit GGUF at 32,768 tokens of context, most experts in RAM (121 GiB of 177 GiB system RAM used). Prefill rises with prompt length to 974 t/s at 26,641 tokens; decode stays 18.7-19.0 t/s across contexts. On 9 GPUs (8x RTX 5060 Ti 16 GB plus the 3090) decode is about 28 t/s and 8K prompts run 1150-1210 t/s. Calibration kept the engine's own settings at 21.6 t/s.

generation:

28.6 tokens/s

quant:

exl3-3.05bpw (EXL3)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 28.55 tok/s decode on an RTX 3090. Setup is ExLlamaV3 with exl3-3.05bpw and 131k context, all-RAM fast mode with CPU offload requiring ~238 GB free RAM. Lab suite passed 6/6 with 28.6 tok/s decode and 989 tok/s prefill; at concurrency 4 the aggregate was 28.62 tok/s (28.45 per stream) because fast mode decodes one stream at a time.

generation:

17.8 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 17.8 tok/s decode and 996 tok/s prefill on an RTX 3090 with the 55 GB NVMe offload launch. Setup is the glm53-flash-offload v4.4-nvme image, with the model's weights streamed from NVMe (55 GB) plus 16 GB resident; the 16 GB launch measured 13.3 tok/s decode and 986 tok/s prefill and failed the 15 tok/s speed gate. Previous runs were 17.3 tok/s for 55 GB and 13.1 tok/s for 16 GB. The 55 GB launch passed all gates and was accepted; the 16 GB launch was reported. Host settings outside the launch hash were --gpus device=0 and --cpuset-cpus 2-39 with no read cap.

generation:

17.3 tokens/s

prompt processing (prefill):

844.0 tokens/s

quant:

3.05bpw (EXL3)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 17.3 tok/s decode on an RTX 3090. Setup is ExLlamaV3 with a 3.05bpw EXL3 quant in NVMe mode (55 GB), 128k context, single card. The context gate read 93,437 prompt tokens at 844 tok/s prefill. An NVMe read cap of 8 GB/s cost about 8% of C1 decode and 26% of prefill; uncapped same-session figures were C1 18.23 and C4 19.59 tok/s. Capped standard table: 8k C1 16.61, C2 14.91, C4 13.10; 32k C1 14.43, C2 15.41.

generation:

21.5 tokens/s

quant:

GSQ-RCO 3.0bit Q4_K attention (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 21.5 t/s decode on 2x RTX 3090 with a VRAM-filling MoE expert cache, up from 13.8 t/s without it. Setup is a neurall/llama.cpp fork with a GSQ-RCO 3.0-bit GGUF (Q4_K attention), 32k context, experts in RAM with a GPU expert cache, on a Ryzen 7 3700X with 125 GB DDR4. Long 12k-token code prompt: 181 t/s prompt processing and 15.7 t/s decode with the cache, versus 217 t/s and 12.3 t/s without. Perplexity 3.5871 on wikitext-2, 0.95% above the original quant's 3.5534.

generation:

69.9 tokens/s

quant:

IQ2_XXS (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 69.89 t/s on an RTX PRO 6000 Blackwell 96 GB. Setup is llama.cpp with an IQ2_XXS GGUF and Q8_0 KV cache, fully resident on the card. Decode falls to 52.74 t/s at 8192 context, 31.70 at 32768 and 12.24 at 131072. The same file on a single RTX 3090 24 GB with 64 GB DDR5 and -ncmoe 40 gives 11.55 t/s decode and 36.69 t/s prefill. The IQ3_XXS build is not benchmarked.

generation:

6.2 tokens/s

prompt processing (prefill):

2.6 tokens/s

quant:

Q4_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash Q4_K at 6.21 t/s decode and 2.60 t/s prefill on an M4 Max 128 GiB via SSD streaming. Setup is ds4 with Q4_K GGUF, 4096 context, 128-token protocol, cold and warm runs. The 177.77 GiB model streams from SSD; GPU time is only ~24 ms of ~161 ms per token (15%), with host-side per-layer expert-cache bookkeeping dominating. Removing the synchronous selected-id readback changed nothing (6.22/6.24 t/s, byte-identical output). MTP was ~15% slower (5.28/5.22 t/s). Cache budgets of 48/56/85 GB gave 5.3/5.3/4.6 t/s; the automatic 70.1 GiB budget was best. All tested flags stayed within noise of 6.2 t/s.

generation:

19.2 tokens/s

quant:

Q2_K Medium (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 19.17 tok/s decode on 2x RTX 3090. Setup is llama.cpp with Q2_K Medium GGUF and F16 KV cache, 65536 context, layer split across two cards with static expert offload to CPU and an external DFlash2 Q4_K_M drafter (SPEC_N=2). Engine-side deep-prefill mean 117.78 tok/s; 10K prefill mean 118.99 tok/s. DFlash fired on 16/16 requests with mean acceptance 0.681. Both cards power-capped at 210 W. Soak-continuous was skipped.

generation:

15.0 tokens/s

prompt processing (prefill):

415.0 tokens/s

quant:

Maya-S

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at ~415 tok/s prefill and ~15 tok/s decode on an RX 7900 XT 20 GB. Setup is Project Maya (v1.0.11, experimental AMD ROCm port) with Maya-S quant, 8K context, ROCm 7.2, 4K-token prompts, greedy sampling; the dGPU box has 192 GB of RAM, with hot experts in VRAM, next in pinned RAM and the rest on NVMe. Other configurations: Radeon AI PRO R9700 32 GB at ~500 tok/s prefill and ~20 tok/s decode; Strix Halo (Ryzen AI Max+ 395, 8060S, 128 GB) at ~210 tok/s prefill and ~18 tok/s decode; R9700 + 7900 XT with layer split and MTP drafting at ~490 tok/s prefill and ~34 tok/s decode. With less RAM more experts come from the SSD and decode drops.

generation:

15-18 tokens/s

prompt processing (prefill):

1000.0 tokens/s

quant:

NVFP4 (NVFP4)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash NVFP4 on one RTX 5090 at ~15-18 tok/s decode and ~1000 tok/s prefill on a 64K-token prompt. Setup is strata-glm, a Strata fork, with the NVFP4 checkpoint on one RTX 5090, experts streamed from 128 GB DDR5 RAM and two NVMe drives. Decode on a ~1-1.4K chat prompt was 18.0 tok/s on Cyrillic chat and 14.8 on code by default, 16.5 and 12.9 in exact mode, and 15.3 in exact mode after the 64K prompt. llama.cpp on the same box managed 2.4-2.9 tok/s decode and 29 tok/s prefill. The project is archived.

generation:

66.0 tokens/s

prompt processing (prefill):

1376.0 tokens/s

quant:

NVFP4 (NVFP4)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3 Flash NVFP4 on two DGX Sparks at a 66.0 t/s mean decode across a task battery, with prefill at 1376 t/s on a 1.5K prompt. Setup is the GLM-5.3-Flash-NVFP4-DFlash2 recipe with DFlash2 speculative decoding, 3.55 of 8 draft tokens accepted per step, single request, and a KV pool of 560,362 tokens. Battery tasks range from 37.1 t/s on prose to 97.2 on bulk SQL inserts, and prefill holds near 1,576 t/s at 32K and 128K. The previous DeepSeek V4 Flash setup averaged 61.5 t/s on the same battery. A 40-minute soak at c4 served 503 requests with 0 errors at 83.6 t/s aggregate.

generation:

95.6 tokens/s

prompt processing (prefill):

1529.0 tokens/s

quant:

3.05bpw (EXL3)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 95.6 t/s decode and 1,529 t/s prefill on an 8K input, running on two CMP 170HX 64GB cards. Setup is ExLlamaV3 1.5.4 with an EXL3 3.05bpw target, DFlash2 EXL3 6bpw K7 drafter and Q8 KV cache, target weights fully resident in HBM at about 116.6GiB across the two cards. Decode stays near 90 t/s out to a 385K input (90.1 t/s, 1,535 t/s prefill); DFlash acceptance was around 87%, against roughly 60 t/s on the older MTP d2 path. The user also compared a Qwen3.8-Flash-Next AWQ INT4 + FP8 PLE vLLM setup on the same machine, which reached 129.5 t/s decode and 5,513 t/s prefill at 8K, but GLM finished three one-shot coding tasks sooner (4m40s vs 5m30s, 10m05s vs 14m40s, 4m40s vs 24m10s).

generation:

19.8 tokens/s

prompt processing (prefill):

437.8 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 19.8 tok/s decode and 437.8 tok/s prefill on one RTX 3090 24 GB with 251 GB RAM. Setup is the Strata engine with a UD-Q4_K_XL GGUF pack, --chunk 4096, 16K-token prompt, tiered expert cache streaming from RAM and NVMe. Decode splits into 12.4 ms expert copies, 0.2 ms expert kernels and 37.2 ms dense/sync per 49.8 ms token. A 64K prompt prefills at 420.7 tok/s; a 1K prompt at 132 tok/s prefill and 19.8 tok/s decode; disk-only tier decodes at 1.15 tok/s. --chunk 8192 OOMs on 24 GB.

generation:

22.0 tokens/s

prompt processing (prefill):

421.1 tokens/s

quant:

IQ4_XS (GGUF)

kv:

f16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports GLM-5.3-Flash at 21.96 t/s decode (narrative) and 26.10 t/s (code) on 2x RTX 5090. Setup is llama.cpp with IQ4_XS GGUF weights, f16 KV cache, 204800-token context, DFlash2 Q4_K_M draft model with speculative decoding (n=3), and MoE expert tensors offloaded to CPU. Prefill measured 421.07 t/s at ~10K prompt tokens and 448.20 t/s at ~87K. Soak test passed with p50 decode 20.70 t/s and 100% TPS retention over 25 turns. Expert cache was requested but not allocated (CACHE_DISABLED), so the run measured a half-cached path. Stress test failed the long-context recall ladder.