Hy4-Flash
on M5 Max 128GB · Kapitsa
Summary
User reports Hy4-Flash at 36.4 t/s decode and 968 t/s prefill on a MacBook Pro M5 Max with 128 GB, using their own Kapitsa engine.
Setup is Kapitsa with a 4-bit MLX quant (8-bit trunk, 4-bit routed experts, group size 64) and the fast profile; the 134 GB pack keeps every expert in RAM with an 88 GB peak, Engram tables memory-mapped on the CPU.
Decode falls from 36.4 t/s at short context to 34.6 t/s at 32K; prefill is 657 t/s at 4K on the default profile and 968-971 t/s with the fast profile. MLX 3.1-bit is compared at 20.8 t/s decode and 228 t/s prefill at 4K. A 48 GB M5 Pro streaming experts from SSD decodes at about 3.5 t/s.