llamaperf

GLM-5.3 320B (18B active) Flash

on M1 Pro 32GB · apus · 1,048,576 ctx

Apple hardware
Oct 3, 2026
Throughput
0.1 t/s gen
Quant
FP8-E4M3 (FP8)
System RAM
32 GB

Summary

User reports GLM-5.3-Flash at ~0.085 tok/s decode (~11-12 s/token) on a MacBook Pro M1 Pro 32 GB. Setup is the apus engine with the official FP8-E4M3 checkpoint, 2 GiB expert cache, --tiered streaming of routed experts from NVMe, MTP speculative decoding on by default (draft depth 3). Prefill was ~76 s for a 26-token prompt, model load ~7 s, peak footprint ~22.3 GiB, weights ~306 GiB, context up to 1,048,576 tokens. The engine reads ~7.9 GiB of experts per generated token cold; the author notes the wall at 32 GB is memory capacity, not disk.