llamaperf

Hy4-Flash

on M5 Max 128GB · Kapitsa

Tone: positiveApple hardware
Nov 20, 2026
Throughput
36.4 t/s gen
Quant
4-bit (MLX)
System RAM
128 GB

Summary

User reports Hy4-Flash at 36.4 t/s decode on a MacBook Pro M5 Max (40-core GPU, 128 GB), with prefill of 657 tok/s at 4K context in the default profile and 968 tok/s with --profile fast. Setup is the Kapitsa engine with a 4-bit MLX quant (8-bit trunk, 4-bit group-64 routed experts, plus Engram tables), 134 GB on disk and 88 GB peak RAM; the Engram tables are memory-mapped and read on the CPU. Decode falls from 36.4 t/s at short context to 34.6 t/s at 32K, while MLX (mlx-lm 0.32.2, 3.1-bit) drops from 20.8 to 4.4 t/s over the same contexts. Warm turns in a cached conversation start in 0.15-0.21 s at 1K to 32K. A 48 GB M5 Pro streams experts from SSD and decodes at about 3.5 tok/s.