llamaperf
Oct 13, 2026
Throughput
20.4 t/s gen
Quant
int4 affine group64 (MLX)
System RAM
24 GB

Summary

User reports Edge0 at 20.4 t/s on a single Apple Mac mini M4 Pro with 24 GB unified memory. Setup is the Edge0 MLX pipeline with int4 affine group-64 weights, a K=4 production profile, and SSD-streamed experts; allocated context is not reported. The figure is the paper's single-device warm decode result, a median over three paired rounds, and the timed output count is absent. The same paper reports prefill at 113 t/s cold and 140 t/s warm.