Edge0 8B-A1B-preview K8 learned routing + unmerged recovery LoRA; LRU64
on M4 Pro 24GB · Edge0 (MLX backend)
Summary
User reports Edge0 at 28 t/s on a single Apple Mac mini M4 Pro with 24 GB unified memory.
Setup is the Edge0 MLX backend with int4 affine group-64 weights, prediction-as-routing staged decode, whole-layer prefill and SSD-streamed experts, using a shared LRU of 64 expert bundles.
The figure is the paper's single-device warm decode result for the released Edge0 8B K=8 low-memory profile, a median over three paired rounds; a separate LRU1024 variant reaches 31.8 t/s with about 0.9 GiB more memory.