llamaperf

DeepSeek V4.1 Flash 552B (16B active)

on M3 Ultra · oMLX · 2,048 ctx

Oct 8, 2026
Throughput
26.5-27.2 t/s gen · 490-500 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
fp8 (MLX)

Summary

User reports a DeepSeek-V4.1-Flash fast path in oMLX reaching ~26.5–27.2 tok/s decode and ~490–500 tok/s prefill at ~2k context on an M3 Ultra. Setup is oMLX with MLX 0.32.2, fp8 weights, wired limit on, Engram stub backend; the stock language.py baseline was ~15.5 tok/s decode and ~337 tok/s prefill. Batched aggregate at B=8 is ~79.5 tok/s (~3x vs B=1); the mmap plus hot cache variant gives ~25.5 tok/s decode and ~481 tok/s prefill.