Local model performance on your hardware

generation:

31.7 tokens/s

quant:

IQ2_XXS/Q2_K (GGUF)

User reports DeepSeek V4 Flash Vision-Exp at 31.7 tok/s decode on a Mac Studio M4 Max 128GB. Setup is ds4 (DwarfStar) with IQ2_XXS/Q2_K GGUF, DSpark speculative decoding at confidence 0.6, and a 32768 context length. Thinking-on pagoda run reached 29.84 tok/s over 26,415 completion tokens; 1M context loaded at 97.15 GiB planned total with 32.0 tok/s decode. User notes DSpark adds roughly 10-20% over no speculation.

generation:

6.2 tokens/s

prompt processing (prefill):

2.6 tokens/s

quant:

Q4_K (GGUF)

User reports GLM-5.3-Flash Q4_K at 6.21 t/s decode and 2.60 t/s prefill on an M4 Max 128 GiB via SSD streaming. Setup is ds4 with Q4_K GGUF, 4096 context, 128-token protocol, cold and warm runs. The 177.77 GiB model streams from SSD; GPU time is only ~24 ms of ~161 ms per token (15%), with host-side per-layer expert-cache bookkeeping dominating. Removing the synchronous selected-id readback changed nothing (6.22/6.24 t/s, byte-identical output). MTP was ~15% slower (5.28/5.22 t/s). Cache budgets of 48/56/85 GB gave 5.3/5.3/4.6 t/s; the automatic 70.1 GiB budget was best. All tested flags stayed within noise of 6.2 t/s.