DeepSeek V4.1 Flash 552B (16B active)
on M3 Ultra · oMLX · 2,048 ctx
Summary
User reports a DeepSeek-V4.1-Flash fast path in oMLX reaching ~26.5–27.2 tok/s decode and ~490–500 tok/s prefill at ~2k context on an M3 Ultra.
Setup is oMLX with MLX 0.32.2, fp8 weights, wired limit on, Engram stub backend; the stock language.py baseline was ~15.5 tok/s decode and ~337 tok/s prefill.
Batched aggregate at B=8 is ~79.5 tok/s (~3x vs B=1); the mmap plus hot cache variant gives ~25.5 tok/s decode and ~481 tok/s prefill.