DeepSeek V4 Flash 284B (13B active)
on 2× 2x Mac Studio 512GB · MLX · 65,536 ctx
Use cases
long-context
Summary
User reports DeepSeek-V4-Flash-0731 at 542 tok/s prefill on 64k context on 2x Mac Studio 512GB.
Setup is MLX with mxfp4 routed experts and affine-8 gs64 elsewhere (155 GB), tensor-parallel over Thunderbolt RDMA, 2 nodes.
Decode is ~26-29 tok/s flat at 64-256k context and ~31 tok/s flat at 6k generated tokens, up from ~10 and 17.7 decaying on the stock baseline. Prefill reaches 510 tok/s at 128k and 366 tok/s at 256k, with 128k prefill transient memory cut from +99 GB to ~24 GB per node. Recall is 5/5 at all depths to 256k. User notes 512k+ cold prefill still exceeds the per-node transient budget beside a resident second model.