Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks oMLX 0.7.0 against 0.7.0rc1 on an M5 Ultra 256GB, running GLM-5.3-Flash oQ4 at 4K–200K context. Prefill improves from 792 to 2,270 tok/s (median) and decode from 55.4 to 76.7 tok/s. A 1M-token prompt completes with prefill 1,485 tok/s, decode 41.5 tok/s, and time to first token 11.8 min. Setup uses Lightning MTP, TurboQuant KV 4-bit, and one model loaded at a time.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM-5.3 Flash at 68.8 tok/s on an M5 Ultra 256GB.
Setup is oMLX 0.7.0 with oQ4e quantization and MTP speculative decoding, with more optimization still to come.
Prefill is 1,878 toks. User shared the result because other benchmarks looked lower than expected.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingvision
User reports GLM-5.3 Flash at ~40 t/s decode and ~1500 t/s prefill on 2x DGX Sparks.
Setup is vLLM with NVFP4 weights and DFlash2 speculative decoding, tensor parallel across 2 cards, 260k context via Claude Code scaffold; decode figure quoted at 100k context.
User says the model feels between GLM 5.1 and 5.3 for coding, has good vision and 3D understanding, and calls it a "good enough" home setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports GLM-5.3-Flash at 69.3 tok/s single-stream coding peak on four NVIDIA DGX Spark (GB10) nodes at tensor-parallel 4.
Setup is vLLM with NVFP4 weights and FP8 e4m3 KV cache, DFlash2 block-diffusion speculative decoding with a 5-layer Qwen3 drafter proposing 7 tokens per step; ~45 GiB of weights per node.
Mixed-prompt sustained decode is 36.7 tok/s at c1 (42.2 tok/s at temp=0), 56.1 tok/s aggregate at c2 and 84.7 tok/s at c4; prefill holds ~1,700 tok/s from 4K to 128K context. A v3 config with pinned KV cache and chunked prefill regressed 25% to 31.5 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM-5.3-Flash at ~0.085 tok/s decode (~11-12 s/token) on a MacBook Pro M1 Pro 32 GB.
Setup is the apus engine with the official FP8-E4M3 checkpoint, 2 GiB expert cache, --tiered streaming of routed experts from NVMe, MTP speculative decoding on by default (draft depth 3).
Prefill was ~76 s for a 26-token prompt, model load ~7 s, peak footprint ~22.3 GiB, weights ~306 GiB, context up to 1,048,576 tokens. The engine reads ~7.9 GiB of experts per generated token cold; the author notes the wall at 32 GB is memory capacity, not disk.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-context
User benchmarks GLM-5.3-Flash on two GB10 nodes (DGX Spark-class, 121 GiB unified memory each, tensor-parallel 2 over RoCE), comparing NVFP4 and EXL3 quantizations. Single-stream decode at 8k context is 30.1 tok/s for NVFP4 and 28.8 tok/s for EXL3.
Setup uses vLLM with DFlash2 speculative decoding (k=7) and 8k context. NVFP4 collapses under concurrency (7.1 tok/s per stream at 4 streams) while EXL3 holds steady (26.7 tok/s).
User reports NVFP4 prefills ~1.8x faster cold (1667.7 vs 925.9 tok/s at 8k) but has zero prefix cache hits, while EXL3 achieves 6.9x faster TTFT on reuse. Agent sessions are warm and concurrent, favoring EXL3. Later updates show k=3 recovers +87% for NVFP4 at C4, and TensorFold TP=2 pairs reach 113.2 tok/s structured single-stream.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-contextvisionagentictool-use
User reports GLM-5.3 Flash at 179.50 tok/s single-stream decode on two RTX PRO 6000 Blackwell 96GB GPUs.
Setup is vLLM with EXL3-K3 quant and FP8 KV cache, TP2 + EP2 + DCP2, DFlash2 speculative decoding, 1,048,576-token context ceiling.
Aggregate decode reaches 1,045.47 tok/s at concurrency 16; prefill is 4,107-4,258 tok/s across 8K-128K prompts. Exact one-million-token six-needle retrieval passed 6/6 in 284.859 seconds.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports GLM-5.3-Flash at 580 tok/s prefill (3.5K) and 546 tok/s at 64K on a single Strix Halo 128GB mini PC (Ryzen AI Max+ 395, gfx1151).
Setup is the Kyojin engine built on ExLlamaV3 with EXL3 weights, 99.7 GB pack, MTP speculative decoding with 2 draft tokens.
Decode is 26 to 30 tok/s with MTP. A second model, MiMo-V2.6-Flash-MOPD (105 GB), reaches about 650 tok/s prefill at 4K and 32 prose / 35 chat / 44 code tok/s with speculation, 29 plain. KLD vs official FP8 is 0.151 for GLM and 0.0713 for MiMo. GLM at 128K context and other GPUs are not measured yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM-5.3-Flash at 63.7 tok/s single-request decode on 4x RTX PRO 6000 Blackwell GPUs.
Setup is an experimental pinned SGLang build with an NVFP4 checkpoint, FP8 E4M3 KV cache, FlashInfer sparse MLA, and 262,144-token context, using tensor parallel TP4/EP4.
With EAGLE speculative decoding (adaptive, up to 5 steps) the single-request figure rises to 82.8 tok/s. Eight concurrent requests give 183.8 tok/s aggregate baseline and 182.1 tok/s with EAGLE.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM 5.3 Flash at around 22 tok/s on a dual DGX Spark setup.
Setup is two DGX Sparks out of a four-Spark cluster.
User found GLM 5.3 overly verbose and slow, and is now running DeepSeek V4 0731 on two Sparks and Qwen 3.8 Flash on the other two.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports GLM-flash-4bit with MTP on an M5 Ultra 512GB, reaching 735.5 prompt t/s and 71.1 generation t/s at 131,072 prompt tokens.
Setup is raw mlx-vlm with --prefill-step-size 8192. At 32,768 tokens the run gave 1056.0 prompt t/s and 72.7 generation t/s; at 65,536 tokens it gave 920.0 prompt t/s and 73.7 generation t/s.
With --prefill-step-size 2048 the same trials gave 623.6 prompt t/s and 50.8 generation t/s at 131,072 tokens. The user also notes omlx with adaptive step size matched 8k performance from 2048 at 64k context and above.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports GLM-5.3 Flash at around 28 t/s across four RTX 3090s at a quant of 3.
Setup is four RTX 3090 24 GB cards on a Threadripper PRO 3975WX with 256 GB RAM, PCIe only, no NVLink.
The user is planning a Qwen 3.8 Flash Next or Qwen 3.8 27B setup and asks for advice on model and hardware configuration.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks GLM-5.3 321B across a MacBook M5 Max 128GB and an AMD Strix Halo 128GB connected over TB4.
The default tensor split was slower than the Mac alone. A Mac-heavy split improved prompt processing from 188 to 333 t/s and generation from 17 to 24 t/s.
User also reports 12-15 t/s for model sizes of 148-200 GB and plans to add two Asus GX10 next week.
User reports Terminal Bench v4 scores for several models, with GLM-5.3 leading at 41.9%.
Other results are GLM-5.3-Flash at 32.8%, DSV4.1-Flash at 26.8%, Qwen3.8-Flash-Next at 25.3%, DSV4-Pro at 14.1%, Kimi-K3 at 12.6%, DSV4-Flash at 12.1%, Qwen3.8-27B at 5.6%, Muse Glimmer at 0.5% and gemma4-31b at 0.0%.
The user describes GLM-5.3 as in a league of its own among open models. No hardware or inference engine details are given.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks MoE streaming in oMLX on an M4 Pro 48GB across four models. GLM-5.3-Flash-JANG-MTP reaches 1.82 tok/s with 12.36s TTFT, 10.52 GiB after load and 14.68 GiB peak. Qwen3.8-JANG 4S reaches 3.63 tok/s with 8.56s TTFT, 7.04 GiB after load and 11.19 GiB peak. Qwen3.8-JANG 4M reaches 3.19 tok/s with 11.23s TTFT, 7.05 GiB after load and 11.11 GiB peak. DeepSeek-V4-Flash-0731-JANG reaches 2.71 tok/s with 6.52s TTFT, 8.25 GiB after load and 16.87 GiB peak.
The primary record uses GLM-5.3-Flash-JANG-MTP. The other models are Qwen3.8 in JANG 4S and 4M quants and DeepSeek V4 Flash in the 0731 variant with the JANG quant.
MoE streaming allows larger MoE models to run with a lower memory footprint at the cost of speed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodingtool-use
User reports GLM-5.3-Flash Q4 at 37.4 t/s end to end at 300k context on an M3 Ultra, against 21.6 t/s for stock ds4.
Setup is a ds4 branch tuned for the M3 Ultra, decoding serially in a single stream without the dflash drafter, with prefill at 550 t/s at 62k.
Serial decode rises from 29 to 40 t/s at short context and from 24 to 38 at 62k. In a claude code harness at ~200k depth it averages over 38 t/s output and reaches 60 t/s writing SQL, where the optional dflash drafter adds +20 to +50 percent.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User runs a 186 GiB MoE model on 96 GB of VRAM, using host RAM for pinned experts, and reports 24.5 t/s decode at 8k context and 27.0 t/s at 0k.
Prefill reaches 159 t/s at 27k. Host RAM is about 208 GB pinned, the CPU does no compute and only feeds experts, and transfers are NUMA-aware. The setup is NVIDIA SM120 only.
Prefix caching with mid-prompt checkpoints cuts TTFT from 67.5s to 18.4s at 8k and from about 923s to 79s at 97k.