Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next Q2 at 86.7 tok/s with MTP on an M5 Max 128GB, using sf-q3-8flash, their stripped-down fork of antirez's ds4 engine.
Setup is the sf-q3-8flash fork of ds4 on Metal with Q2 weights, MTP speculative decoding on, the whole model in unified memory. Stock ds4 ran the same Q2 MTP setup at 75.8 tok/s; on Q4 MTP went from 77.8 to 85.9 tok/s. No plain decode figure is given, only a 9-13% decode gain on Q2 and 2-11% prefill gain on Q4.
Output stays bit-exact against stock ds4. An optional SSD streaming mode for experts on a simulated 48GB machine ran Q2 at 27 tok/s, around 35 tok/s with MTP.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3 30B-A3B at 130.628 t/s generation and 5402.434 t/s prompt processing on an Apple M5 Max with 128GB unified memory.
Setup is MLX 0.32.2 with MLX-LM 0.31.3 on macOS 27.0; quantization, context length and KV cache settings are not reported.
Figures come from a third-party hardware review (Tbreak, September 21, 2026), not a reproducible benchmark log; prompt and generated token counts are unknown.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4.1 Flash at Q2 running on an M5 Max 128GB Mac, with a fork of ds4 reaching 24.4 t/s generation at 2048 context versus 12.1 t/s for stock ds4.
Setup is ds4 with a 341 GB GGUF (152 GB weights plus 189 GB Engram tables) streamed from SSD, no KV quant, bit-exact output. The fork doubles steady decode to 25.7 t/s at 2048 context and 22.0 t/s at 32768 context, and prefill from 16k to 32k context rises from 404 t/s to 636 t/s.
Adding a byte-identical external Thunderbolt 5 SSD copy cuts time to first token on a 3.5K prompt from 13.2 s to 11.2 s and after an 8K context from 1.38 s to 0.18 s; decode is unchanged. The enclosure runs at PCIe 4.0 x4, about half the internal SSD speed. A two-SSD attempt in upstream ds4's mmap path was bit-exact but 13-38% slower on prefill and was not submitted.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen 3.8 27B at 39.9 tok/s with DFlash 2 speculative decoding on an M5 Max 128GB, versus 17.9 tok/s plain decode.
Setup is mlx-dspark 0.14 on MLX 0.32 with the 8-bit MLX build (~29 GB) and 4-bit/8-bit quantized KV cache; the DFlash 2 drafter is incoai/Qwen3.8-27B-DFlash2 (3.8 GB).
In the agent harness the model runs 20-29 tok/s. A bf16 run of the same model measured 9.7 tok/s plain and 36.5 tok/s with DFlash 2 (4.43 accepted tokens per round, 137 target forwards, output identical). Agent-12 scores with DFlash 2 were 12/12 easy in 122 s and 7/8 hard in 401 s, about 3x faster than the Aug 18 run at the same accuracy. The Claude Code smoke test passed in 38 s. Gemma 4 31B Abliterated 4-bit ran at 26 tok/s and Gemma 4 E4B 4-bit failed the smoke test. On an M4 Pro 64 GB, Gemma 4 31B runs at ~13.5 tok/s and server-side prompt fixes cut hello latency from ~120 s to ~3-5 s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Kimi K3 (2.78T) at 0.96 tok/s on a MacBook Pro M5 Max 128GB, with the trunk in memory and 1.44 TB of 4-bit routed experts streamed from NVMe drives.
Setup is a llama.cpp-derived fork with experts split across the internal SSD plus up to three Thunderbolt 5 enclosures, each holding a byte-identical replica.
Decode scales with drive count: 0.55 tok/s on one drive, 0.75 with one enclosure, 0.89 with two, 0.96 with three (1.75x), and 1.00 steady over 512 generated tokens, 1.13 over 128.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen 3.8 27B AXQ MXFP4 MTP at 76.70 tok/s decode and 792.4 tok/s prefill on an M5 Max 128 GB.
Setup is AX Engine 7.5.3 with MXFP4 MTP weights, median of 20 runs on the flappy suite.
The same campaign measured the 6-bit pack at 76.04 / 768.7 and long_code at 72.71 / 856.9; MTPLX 2.11.2 reached 63.91 / 697.3 and mlx-lm 34.20 on the MXFP4 pack.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codinglong-contextvision
User reports Qwen3.8 Flash-Next at 73.5 tok/s with MTP speculative decoding on an M5 Max 128GB, versus 43.8 tok/s plain autoregressive.
Setup is MTPLX with a dynamic 4-bit quant (8-bit attention) MLX pack, 262,144-token context, MTP depth ceiling 3, single stream through mtplx serve.
Prompt processing on a 4,061-token prompt is 1,453 tok/s on MTPLX 2.12.0 (786 tok/s on 2.11.3); a 65,502-token prompt reads at 1,094 tok/s with 63.5 tok/s decode. The pack downloads at 115.1 GB including a 32 GB n-gram table streamed from SSD, peaks at 87 GiB while serving, and is recommended for Macs with 128 GB or more.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.6-35B-A3B at 105 tok/s decode at 128 context and 97 tok/s at 8k on an M5 Max 128 GB.
Setup is llama.cpp with Q8_0 GGUF weights, MTP speculative decoding on, 65536 context, 45 GB memory.
Without MTP the same model gives 93 tok/s at 128 and 87 tok/s at 8k; MLX gives 85 and 79 tok/s. MTP adds 75% on the dense 27B (17 to 32 tok/s) but only 12% on the MoE 35B-A3B. llama.cpp beats MLX by 10-24% on this machine. DeepSeek-V4-Flash under ds4 at q2-q4 (91 GB) reaches 33 tok/s at 128 and 28 tok/s at 8k.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B at 47.0 t/s generation and 854 t/s prompt processing on an Apple M5 Max with 128 GB unified memory.
Setup is mlx-vlm with the 4-bit MLX quantisation and MTP speculative decoding enabled, 15.5 GB resident. The 8-bit quant reaches 28.7 t/s generation and 883 t/s prompt processing at 28.1 GB resident.
MTP speculative decoding is reported as worth 2.2x (15.5 t/s to 34.1 t/s at 8-bit). Decode falls to 36.1 t/s at 8K context, and time to first token climbs to about 14 s at that depth.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 1900 t/s prefill and 90 t/s generation on an M5 Max 128GB.
Setup is mlx-serve with Sushi-4 quant, an MLX-EXL3 hybrid, with MTP and Vision supported.
User also cites M1 Max 64GB at ~350 t/s prefill and 30 t/s gen, and M5 Pro at ~900 t/s prefill and 60 t/s gen.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 50-70 t/s on an M5 Max 128GB after upgrading to macOS 27, down from 60-90 t/s on macOS 26.
Setup is oMLX with oQ5e quantization.
The user reports roughly 20% slower token generation after the OS upgrade and asks whether others see the same.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at ~26 tok/s generation and ~190 tok/s prompt processing on a MacBook Pro M5 Max with 128 GB unified memory.
Setup is LM Studio with antirez's deepseek_v4 llama.cpp fork sideloaded, IQ2XXS GGUF weights, F16 KV cache, and the full 1048576 token context. The model is a 256-expert MoE with ~80 GB resident.
The figures are approximate and the run is a workaround: upstream llama.cpp does not yet support the deepseek_v4 architecture and LM Studio's MLX runtime rejects V4 GGUFs. Long prompts beyond 16k tokens are flushed in 16k chunks due to the n_batch cap.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a Swift-Qwen3.8-27B 4bit model running on the Splash engine on an M5 Max 128GB, with decode of 79–129 tok/s.
At 64k context the model reaches 87.8 tok/s versus 36.6 tok/s for the same weights in oMLX (oQ4e-mtp, mixed 4/5-bit), and the gap widens with context.
The user notes Splash's /v1 defaults to greedy while Swift wants temp 1.0 / top_k 20 / top_p 0.95, and that the format cannot carry generation_config.json.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports Nex-N2.5-mini at 133.6 t/s on an Apple M5 Max.
Setup is MLX with the MLX-4bit quant, using temperature 0.7, top_p 0.95, top_k 40, and reasoning_effort high.
User calls the generation speed and prompt processing really good, the memory footprint okay, and the quality good across the board, and plans to use it for agents and possibly coding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks GLM-5.3 321B across a MacBook M5 Max 128GB and an AMD Strix Halo 128GB connected over TB4.
The default tensor split was slower than the Mac alone. A Mac-heavy split improved prompt processing from 188 to 333 t/s and generation from 17 to 24 t/s.
User also reports 12-15 t/s for model sizes of 148-200 GB and plans to add two Asus GX10 next week.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingsummarization
User asks about a Mac Studio M5 Max 128GB for local Qwen3.8-flash-next.
The user mentions the oQ4e quant at ~55 tok/s and also the oQ5e quant.
The user is debating whether to buy now or wait for the M7 Ultra, for private documents, notes, coding and general assistant work.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-contextcoding
User reports about 40 t/s on prose and 75 t/s on coding at roughly 760k context.
Setup uses 8-bit dense layers and 4-bit expert layers, with peak memory around 117 GB and a requirement of iogpu.wired_limit_mb=120000. Model weights are at ddalcu/Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User benchmarks a 128 GB M5 Max MacBook Pro across multiple models, selecting Qwen 3.6 27B MLX Q8 as the primary one.
Other models tested include Gemma 4, MiniMax M2.7 and DeepSeek V4 Flash.
The user is enthusiastic about local AI.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks several models on an M5 Max 128GB and selects Qwen3.6 35B A3B as the primary model for its MoE detail.
Other models tested include Gemma 4, Qwen 3.6 27B, MiniMax M2.7, Mistral Medium 3.5, Step 3.7 Flash and DeepSeek V4 Flash.
Results show strong performance for MoE models.
User compares DeepSeek-V4-Flash on an M5 Max MacBook with 128 GB against 2x DGX Spark. Terminal-Bench 2.1 scores 54% (47/87) on the Mac and 52% (45/86) on the Spark pair.
The Mac runs a heavily quantized GGUF at IQ2_XXS/Q2_K, about 2.45 bits per weight. The Spark pair runs the native FP8/FP4 checkpoint.
The Mac wins 11 tasks outright and the Spark pair 9, with no significant difference in task-solving. The Spark pair offers higher speed at 54-58 tok/s single-stream and 253 tok/s aggregate at concurrency 8, plus a larger 262K server context window.