Llama 3.1 405B
on M3 Ultra 512GB · MLX · 1,024 ctx
Oct 8, 2026
Summary
User benchmarks Llama 3.1 405B at 2.99 t/s on a single M3 Ultra 512GB node.
Setup is MLX with Q4 quantisation at 1K context, single node.
This is part of a 200+ configuration campaign across 5 models and 9 topologies on a 5-node M3 Ultra cluster; the report also covers Qwen 2.5 32B, Mixtral 8x7B, DeepSeek V3.2 and Kimi K2.5, with TP and PP scaling results.