M5 Max 128GB

reported speed:

133.6 tokens/s generation

quant:

MLX-4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Nex-N2.5-mini at 133.6 t/s on an Apple M5 Max. Setup is MLX with the MLX-4bit quant, using temperature 0.7, top_p 0.95, top_k 40, and reasoning_effort high. User calls the generation speed and prompt processing really good, the memory footprint okay, and the quality good across the board, and plans to use it for agents and possibly coding.

reported speed:

24.0 tokens/s generation · 333.0 tokens/s prompt processing

quant:

IQ1_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks GLM-5.3 321B across a MacBook M5 Max 128GB and an AMD Strix Halo 128GB connected over TB4. The default tensor split was slower than the Mac alone. A Mac-heavy split improved prompt processing from 188 to 333 t/s and generation from 17 to 24 t/s. User also reports 12-15 t/s for model sizes of 148-200 GB and plans to add two Asus GX10 next week.

reported speed:

55.0 tokens/s generation

quant:

oQ4e

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User asks about a Mac Studio M5 Max 128GB for local Qwen3.8-flash-next. The user mentions the oQ4e quant at ~55 tok/s and also the oQ5e quant. The user is debating whether to buy now or wait for the M7 Ultra, for private documents, notes, coding and general assistant work.

reported speed:

40.0 tokens/s generation

quant:

mixed-4-8bit

kv:

8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports about 40 t/s on prose and 75 t/s on coding at roughly 760k context. Setup uses 8-bit dense layers and 4-bit expert layers, with peak memory around 117 GB and a requirement of iogpu.wired_limit_mb=120000. Model weights are at ddalcu/Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit.

reported speed:

17.2 tokens/s generation · 706.0 tokens/s prompt processing

quant:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks a 128 GB M5 Max MacBook Pro across multiple models, selecting Qwen 3.6 27B MLX Q8 as the primary one. Other models tested include Gemma 4, MiniMax M2.7 and DeepSeek V4 Flash. The user is enthusiastic about local AI.

reported speed:

79.4 tokens/s generation · 2153.0 tokens/s prompt processing

quant:

Q8 XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks several models on an M5 Max 128GB and selects Qwen3.6 35B A3B as the primary model for its MoE detail. Other models tested include Gemma 4, Qwen 3.6 27B, MiniMax M2.7, Mistral Medium 3.5, Step 3.7 Flash and DeepSeek V4 Flash. Results show strong performance for MoE models.

quant:

IQ2_XXS/Q2_K (GGUF)

User compares DeepSeek-V4-Flash on an M5 Max MacBook with 128 GB against 2x DGX Spark. Terminal-Bench 2.1 scores 54% (47/87) on the Mac and 52% (45/86) on the Spark pair. The Mac runs a heavily quantized GGUF at IQ2_XXS/Q2_K, about 2.45 bits per weight. The Spark pair runs the native FP8/FP4 checkpoint. The Mac wins 11 tasks outright and the Spark pair 9, with no significant difference in task-solving. The Spark pair offers higher speed at 54-58 tok/s single-stream and 253 tok/s aggregate at concurrency 8, plus a larger 262K server context window.

reported speed:

31.8 tokens/s generation

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks Qwen3.8-27B on an M5 Max 128GB across four engines. Setup is llama.cpp with native MTP at 31.8 t/s sustained and 22.7 t/s at 56k context, TTFT 125ms, prefix reuse 0.80, RAM 36.8GB; oMLX 0.6 at 29.8 t/s and 24.6 t/s at depth, TTFT 1968ms, prefix reuse 0.82, ~17GB; mlx-vlm with MTP at 43.3 t/s and 28.9 t/s at depth, TTFT 9982ms, no prefix reuse, 15.6GB; and MTPLX 2.6 at 27.1 t/s and 23.1 t/s at depth, TTFT 13400ms, no prefix reuse. The fastest engine, mlx-vlm, is unusable for agents due to 10s TTFT and no prefix reuse. Without --spec-type draft-mtp, llama.cpp drops the MTP head; a draft cap of 4 gives 31.8 t/s versus 21.9 t/s with no speculation, cap 2 from Qwen3.6 presets gives only 22.2 t/s, and cap 6 loses gains. At 262k context with q8_0 KV cache llama.cpp gives 29.6 t/s and 18.5 t/s at depth; at 131k with f16 it gives 31.8 t/s and 22.7 t/s at depth. The model does 5 searches then calls more tools instead of writing the brief and dies at max_turns. reasoning_effort 'none' throws an exception on 3.8, so enable_thinking: false is needed; oMLX does not activate MTP on 3.8; the model field of the response should be checked rather than /v1/models; and the M5 Max throttles to 50% after 80-110s. No generational verdict versus 3.6 is given.

reported speed:

11.5 tokens/s generation · 333.0 tokens/s prompt processing

quant:

UD-Q2_K_XL (gguf)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 79 GB in 2-bit running at 350K context for 3.5 hours on a 128 GB M5 Max, over 100 turns. Prefill runs 1,561 t/s at 5.6K context tapering to 318 t/s at 111K, with normal turns at 77-854 t/s from prefix reuse. Decode is about 30-35 t/s at small context down to 11.5 t/s at 169K, with a dip to 7.7 t/s at around 140K due to Low Power Mode. Decode figures are effective throughput with speculative decoding. The user calls the experience strong for the first ~100K context, then reports role confusion on long-tail tasks, suspecting the 2-bit quant or preview-model long-context quality.

reported speed:

36.6 tokens/s generation

quant:

oQ4e-mtp

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks Qwen3.8-27B MTP quants on an Apple M5 Max. The oQ4e-mtp quant reaches 36.6 t/s with a quality score of 86.8. User calls oQ4e-mtp the best balance of speed and quality and recommends it as the sweet spot.

reported speed:

33.9 tokens/s generation

quant:

Q4_K_S (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks prompt processing at 65,536 context, with throughput falling from 62.80 t/s at 128 context to 33.92 t/s at 65,536. Memory peaked at ~120+ GB, and the system stayed sluggish but usable.

reported speed:

32.4 tokens/s generation

quant:

UD128 (gguf)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Tencent HY3 295B-A21B MoE at 32.4 t/s generation at empty context and 16.3 t/s at 16K context on Metal. Setup is llama.cpp with the UD128 quant (107 GB), a q8_0 KV cache, and 24K context. With MTP n=2, generation peaks at 38 t/s. Prefill runs 528 t/s at pp512 empty and 124 t/s at pp512 at 16K. User compares the result favorably to DeepSeek V4 Flash.

quant:

WinterMix58 (mlx)

User reports a new MLX quantization method, WinterMix, for Qwen3.5-122B-A10B, with two builds: WinterMix58 at 82 GiB and WinterMix48 at 68 GiB. Benchmarks show perplexity improvements over existing MLX quants. The user is enthusiastic about the results and the method's advantages for agentic workflows on Apple Silicon.

reported speed:

7.5 tokens/s generation

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Gemma 4 at 7.5 t/s and Qwen3.6-27B locking up, while Qwen3.6-35B-A3 is fast. User suspects a bug affecting dense models.

reported speed:

5.5 tokens/s generation · 160.0 tokens/s prompt processing

quant:

Q8 (mlx)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports 160 t/s prefill and 5-6 t/s generation, later 4-5 t/s, on an M5 Max 128GB with Qwen 3.6 27B Q8 MLX at 290k context. GPU utilization is 36-50%. User feels performance is lower than expected and asks for comparison.

reported speed:

7.5 tokens/s generation

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports dense models running poorly on an M5 Max 128GB, with Gemma4-31B at about 7.5 t/s and Qwen3.6-27B locking up, while Qwen3.6-35B-A3B MoE is fast. The user mentions using DFLASH, likely flash attention.

reported speed:

5.5 tokens/s generation · 160.0 tokens/s prompt processing

quant:

Q8 (mlx)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports 160 t/s prefill and 5-6 t/s generation on an M5 Max 128GB with Qwen 3.6 27B Q8 MLX at 290k context. GPU utilization sits at only 36-50%. The user finds the generation rate low against an expected 8-14 t/s and asks others for comparison.

reported speed:

7.5 tokens/s generation

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Gemma4-31B at 7.5 t/s and Qwen3.6-27B locking up on an M5 Max with 128 GB, while Qwen3.6-35B-A3 is fast. Setup uses DFLASH.