llamaperf

DeepSeek V4 Flash 284B (13B active)

on 2× M3 Ultra 512GB · MLX · 13,900 ctx

Apple hardware
Oct 3, 2026
Throughput
45-47 t/s gen · 1012.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
4-bit (MLX)
System RAM
512 GB

Summary

User reports DeepSeek-V4-Flash-0731 at 4-bit served across two Mac Studio M3 Ultra (512GB) boxes: prefill 1012 tok/s on a 13.9K-token prompt (TTFT 13.7 s), decode 45-47 tok/s single-stream and 112-117 tok/s aggregate at batch 8. Setup is MLX with an oMLX 0.6.3rc2 serving overlay, MTP/speculative machinery enabled, over a single Thunderbolt 5 cable using the jaccl RDMA mesh backend. The decode figure is given only as a range, so no single generationTps is recorded. The repo is a reproducibility playbook; the oMLX 0.6.3rc2 build artifact is no longer available and must be obtained upstream.