llamaperf

DeepSeek V4 Flash 284B (13B active)

on 2× 2x Mac Studio 512GB · MLX · 65,536 ctx

Tone: positive
Oct 9, 2026
Throughput
26-29 t/s gen · 542.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
mxfp4 (MLX)
System RAM
512 GB

Use cases

long-context

Summary

User reports DeepSeek-V4-Flash-0731 at 542 tok/s prefill on 64k context on 2x Mac Studio 512GB. Setup is MLX with mxfp4 routed experts and affine-8 gs64 elsewhere (155 GB), tensor-parallel over Thunderbolt RDMA, 2 nodes. Decode is ~26-29 tok/s flat at 64-256k context and ~31 tok/s flat at 6k generated tokens, up from ~10 and 17.7 decaying on the stock baseline. Prefill reaches 510 tok/s at 128k and 366 tok/s at 256k, with 128k prefill transient memory cut from +99 GB to ~24 GB per node. Recall is 5/5 at all depths to 256k. User notes 512k+ cold prefill still exceeds the per-node transient budget beside a resident second model.