DeepSeek V4.1 Flash 552B (16B active)
on M5 Ultra 256GB · oMLX · 16,384 ctx
Summary
User reports DeepSeek V4.1 Flash (10-17 checkpoint) running on a 256GB M5 Ultra (64-core GPU, 30-core CPU) at 20-25 tok/s generation and 500 tok/s prefill.
Setup is oMLX with an oQ4e MLX build, 65% MoE expert offload, CED prefill on, SSD N-gram offload on and a 16384 prefill chunk size; the 4-bit N-gram table was swapped for the original FP8 shards.
Single-request tables give 18.8-30.8 tok/s decode on an English novel and 21.2-25.9 tok/s on Python code, with peak memory up to 203.6 GB; context benchmarks show 532 tok/s prefill at 128k and 516 tok/s at 256k.