llamaperf

GLM-5.3 320B (18B active) Flash

on 2× M5 Ultra 256GB · MLX · 17,500 ctx

Tone: positiveApple hardware
Oct 14, 2026
Throughput
36.8 t/s gen
Quant
4-bit (MLX)

Use cases

long-context

Summary

User reports GLM-5.3-Flash 4-bit with MTP at 36.8 tok/s decode on a disaggregated setup pairing an M5 Ultra 256GB (prefill) with an M3 Ultra 512GB (decode). Setup is MLX with the JACCL RDMA backend over two Thunderbolt 5 cables, mlx-vlm 0.7.6, a 17.5k-token prompt and 256 output tokens. Prefill runs on the M5 at 988 tok/s (17.8 s), the 358 MB cache is handed off over TB5 in 38 ms, and first token arrives on the M3 about 18.5 s after the request instead of about 40 s. Single-node figures: M3 Ultra 512GB prefill 438 tok/s at 2k chunks and 454 tok/s at 8k chunks, decode 32.9 tok/s; M5 Ultra 256GB prefill 1,046 tok/s at 2k chunks and 1,188 tok/s at 8k chunks, decode 47.2 tok/s. The user notes the M5 alone decodes faster for a single request, and that the prefill node needs the full model so this only works for models that fit in 256 GB.