Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User benchmarks Nanbeige4.3-33B (dense, 4-bit) on a MacBook Pro M5 Max with 128 GB, comparing mlx-lm, llama.cpp and Kapitsa at prompt lengths from 512 to 32K tokens. At 512 tokens mlx-lm prefills at 757 t/s and decodes at 28.8 t/s, llama.cpp at 691 t/s and 27.3 t/s, and Kapitsa at 693 t/s and 34.9 t/s.
mlx-lm and Kapitsa read the same 4-bit MLX AWQ pack (group 64); llama.cpp reads Q4_K_M. Kapitsa's decode figures use KFlash, its tree-based speculative decoding with a small drafter, while mlx-lm and llama.cpp run plain decoding.
On a 512-token Python coding task Kapitsa reaches 36.6 tok/s greedy and 32.3 tok/s sampled, against 27.1 and 26.7 for mlx-lm and 24.9 and 24.8 for llama.cpp. At 29.5K context mlx-lm has the fastest cold TTFT at 59.4 s, Kapitsa the fastest follow-up TTFT at 300 ms and the lowest peak memory at 28.1 GB. The author works on Kapitsa and notes the KV cache is large, 26 GB for 128K tokens.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-useagentic
User reports Granite Switch 4.1 8B at 56 t/s chat on an M5 Max 128GB, with the 30B at 17 t/s.
Setup is mlx-lm with a custom granite_switch package, 8-bit MLX weights, greedy decoding, one request at a time, medians of 3 runs. The 4-bit builds reach 98 t/s (8B) and 32 t/s (30B).
Decision steps take 0.62 s on the 8B and 2.3 s on the 30B at a ~1.2K-token prompt. Accuracy on 500 labelled claims is 0.814 (8B 8-bit), 0.844 (30B 8-bit) and 0.840 (30B 4-bit), against 0.812 and 0.846 for the bf16 conversions. The user picks the 8B 8-bit for routing and classification.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running Qwen3.6-35B-A3B (Splash build) on an M5 Max 128GB MacBook Pro in LM Studio at roughly 375-400 tok/s with a 262k context.
Setup is LM Studio with the Splash build at 262k context on the 128GB machine.
The speed is given only as a range, so no single figure is recorded. The post's main subject is a cloud agent running a training-and-grading program against the local model, not a benchmark of it.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
math
User reports a Qwen3.6-35B-A3B Splash build in LM Studio on an M5 Max 128GB MacBook Pro at roughly 375-400 tok/s, used to generate and cold-solve 800 math problems.
A Qwopus 122B MLX build at about 38 tok/s acts as judge for disputed items; the cloud agent Muse orchestrates over Tailscale.
Of 800 problems, 731 verified clean, 62 had corrected answer keys, 7 were excluded, and 100 were held out; the solve-and-adjudication passes logged 4.1M+ completion tokens. The user notes the 122B judge went 1-for-2 on the hardest disputes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports NVIDIA Nemotron-Labs Dory 1.0 4B at 101 tok/s on an M5 Max 128GB with one loop, dropping to 69 tok/s at 2 loops and 39 tok/s at 4 loops.
Setup is an MLX port with the 8-bit 4.2 GB build, medians of repeated runs; peak memory is 7.7 GB at 32K context with one loop.
After a 32K prompt the same runs give 82, 54 and 31 tok/s, with prefill at 3,200, 1,980 and 1,070 tok/s. Batch generation reaches about 450 tok/s total across 16 requests at 2 loops, and a 16 GB M1 Pro measured about 28 tok/s at 1K and 19 tok/s at 32K in 4-bit at 2 loops. 8-bit WikiText-2 perplexity falls from 12.18 to 11.77 to 11.70 as loops increase.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-context
User compares Strata against oMLX running Qwen3.8-Flash-Next on a 128GB M5 Max, with Strata decoding at 74 tok/s versus 57.5 tok/s for oMLX at 245K context.
Strata is a one-month-old engine built for this one model, running on Metal.
The comparison holds one request at a time only, which the user notes as the current limitation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 140 tok/s on an M5 Max MacBook Pro with 128GB, the fastest run of their open source Apple Silicon engine kapitsa.
Setup is kapitsa with a 4bit model and kflash speculative decoding, where the 27B verifies a tree of drafter guesses in one pass rather than a single chain.
The user is asking Mac Studio owners to run the same model for comparison.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at roughly 100 tok/s (90-110 through the server) on an M5 Max 128GB with the new ijah engine.
Setup uses EXL3 trellis weights at 5b-6/8b, about 6% behind oMLX but with 5-bit experts.
User says the quant is about 4-5x closer to bf16 than Jundot's oQ4e (KLD 0.033/0.009/0.015 vs 0.128/0.036/0.078) and about 2x closer than Sushi-4bpw. With Inco's DFlash2 drafter the user reports about 120 tok/s but has not verified it yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 50.3 t/s generation and 1084 t/s prompt processing.
Source evidence (verbatim):
Source: https://omlx.ai/benchmarks/performance/ogokptif
# Qwen3.8-27B-oQ5e-mtp on M5 Max (40c) - oMLX Benchmark
Hardware: Apple M5 Max (40-core GPU), 128 GB unified memory. Engine: oMLX v0.7.1.dev1. Model: Qwen3.8-27B-oQ5e-mtp, 5-bit. Multi-token prediction (MTP) is enabled with adaptive max depth 3, so this is a speculative decoding run; DFlash and SpecPrefill are disabled. Single request. At 8k context: 1,084 tokens/s prompt processing and 50.3 tokens/s token generation. At 4k context: 1,137 prompt and 58.4 generation; at 1k: 895.8 prompt and 54.6 generation; at 16k: 882.8 prompt and 75.5 generation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B at 154 t/s decode on a 1k prompt with oMLX's native engine on an M5 Max 128GB, falling to 55 t/s at 262k context.
Setup is oMLX 0.7.1rc1 with the oQ5e-mtp quant, Lightning MTP on, TurboQuant KV off, and a 128-token greedy generation at each context step. Splash 1.3.2 with int8 KV decoded 147 t/s at 1k and 78 t/s at 262k, and used 30.2 GiB against native's 33.5 GiB prefill peak and 37.4 GiB decode.
TurboQuant 5- and 6-bit decoded at 27-29 t/s at 262k, about half of off. The user also ran Qwen3.8-27B, where Splash int8 was faster at almost every context (69 vs 48 t/s at 1k) and native TurboQuant cost 7-25% decode speed with MTP on. No setting showed a detectable accuracy loss of more than about 2 points.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running DeepSeek v3.1 671B Terminus at 3.6 tokens/sec on a MacBook Pro with M5 Max 128GB and 8TB storage.
Setup uses a heavily modified slipstream llama.cpp base that caches MoE experts to disk and calls them in as they change, with a -0.1 bias in the MTP router for in-RAM experts. A custom GGUF-derived format uses BF16 for fixed layers and INT4 for expert layers.
The user says it is working extremely well and generating a somewhat usable 3.6 tokens/sec.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B at 198.6 tok/s generation and 3657.8 tok/s prompt processing on an M5 Max 128GB.
Setup is oMLX 0.7.1dev2 with oQ5e quant, Lightning MTP speculative decoding, and TurboQuant off, at 1024-token prompt and 128-token generation.
Splash engine reaches 182.0 tok/s generation and 6040.8 tok/s prompt processing on the same test; continuous batching at 4x reaches 275.2 tok/s aggregate generation. A second model, Qwen3.7-27B, is also benchmarked.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 79.1 t/s generation.
Source evidence (verbatim):
Source: https://bestllmfor.com/guides/macbook-pro-m5-max-128gb-ai-benchmark/
# MacBook Pro M5 Max 128GB AI benchmark
Joel Ramat ran these tests on his own 16-inch MacBook Pro on September 16, 2026. M5 Max, 18 CPU cores, 40 GPU cores, 128 GB unified memory advertised at 614 GB/s, macOS 27.0, Ollama 0.34.1, plugged in, default GPU memory limit. All figures are raw outputs from Ollama. Each model was run twice with the same prompt (a 300 word explanation of unified memory versus VRAM) and only the second pass kept, one model loaded at a time. Generation speed: gpt-oss 120B MXFP4 (65 GB) 79.1 tok/s; gpt-oss 20B MXFP4 (13 GB) 113.4 tok/s; Qwen 3.6 35B-A3B MLX (23 GB) 167.2 tok/s; Qwen 3.8 27B MLX (18 GB) 68.0 tok/s; Qwen 3.8 27B GGUF Q4 (17 GB) 36.6 tok/s; Gemma 4 12B GGUF Q4 (7.6 GB) 58.1 tok/s. Prefill on a 16,689 token document with gpt-oss 120B: 1,388 tok/s, time to first token 12.0 s, generation 67.6 tok/s on that run. Only short bursts were tested, one machine, one operator.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 oQ4e-mtp restored-prefix B3 / PR1 61bd8a06 at 8.4 t/s on an M5 Max 128GB.
Setup is oMLX with oQ4e 4-bit weights and TurboQuant 4-bit KV cache, three greedy requests batched at roughly 20k / 45k / 135k prompt tokens with SSD prefix cache enabled.
The 8.4 t/s is the per-row rate at the 20k row, measured at 8.35 t/s; the other rows ran 8.83 and 7.84 t/s per row, and the batch phase took 66 seconds.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4.1 Flash at ~30 tok/s decode at 2K context on an M5 Max 128GB, streaming a 341 GB Q2 GGUF from SSD.
Setup is a fork of antirez's ds4 engine (sf-ds4-1flash) with Metal backend, one machine, weights streamed from disk.
Decode is ~26 tok/s at 32K context and settles around 24 tok/s after an hour of heavy use. First token after an 8K prompt dropped from 1.23 s to 0.15 s. Prefill improved 45-99% and decode 62-78% versus upstream ds4. Output is bit-identical to ds4.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3 30B-A3B at 144.4 t/s on an M5 Max with 128GB unified memory.
Setup is MLX with 4-bit affine quantization, group size 64, standard 8 routed experts per token, batch 1 with a 944-token prompt and 512 output tokens.
The figure is the arithmetic mean of six fresh-process runs, ranging 144.10 to 145.01 t/s; an altered 6-expert routing gave 155.61 t/s but is not a standard-model result.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at 36.1 t/s generation and 498.3 t/s prefill on a MacBook Pro M5 Max 128GB 40 GPU.
Setup is oMLX 0.5.7 with a 2.44 bpw mixed MLX quantization (92.8 GB on disk) at 1k prompt, single request, 128 generated tokens, speculative decoding off.
Generation holds 33.8 t/s at 4k and 31.5 t/s at 32k, with peak memory rising 1.03 GB across an 8x context jump; prefill at 32k takes 97 seconds to first token. Continuous batching reaches 83.2 t/s at batch 8 (2.30x).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
vision
User benchmarks Gemma 4 E2B at 205 t/s decode at 4k context on an Apple M5 Max with 128 GB unified memory.
Setup is mlx-vlm 0.4.4 with MLX 0.31.1, 4bit weights, and TurboQuant 2.5-bit KV cache quantization; the model uses 4.7 GB at 4k.
Decode falls to 78 t/s at 256k context. TurboQuant gives +15-19% at 128-256k on larger models but is negative on E2B. The same report also covers Gemma 4 E4B, Gemma 4 26B-A4B, Gemma 4 31B, Qwen 3.5 2B and Qwen 3.5 4B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next-Heretic Q8_0 at 20.45 tok/s on an M5 Max 128GB.
Setup is llama.cpp (closed PR #27739 fork) with Q8_0 and Q4_0 KV cache, -ngl 16, 4096 context, 45.1 GiB wired, MTP draft head enabled.
Measured on 12 varied prompts with 24 warm-up queries discarded. The model is 122B total / 10B active MoE, 197 GB across 5 shards. A 262144-token server ran on llama.cpp v0.6.0 with ~32 tok/s prefill and decode falling to ~3.6 tok/s at 10k tokens. Under memory pressure a larger GPU tier wins: -ngl 24 at 10.29 tok/s with 15 GiB held by other work versus -ngl 6 at 5.75 tok/s with 45 GiB held. Contiguous layer placement beats scattered expert placement by 28.6%. MTP acceptance is 51-83%, helping +44% when experts are cached in RAM but costing 16-33% when they stream from SSD. Stock llama.cpp was 11-19% faster than the fork at 4k/64k/256k, and v0.6.0 another ~7% faster. No quality measurement was run for this file.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 70+ tok/s generation on an M5 Max 128GB MacBook Pro.
Setup is the MLTF engine with 8-bit weights and an INT8 KV cache, using Metal TensorOps W8A8 kernels.
Cold prefill is 1,000+ tok/s on a 2Ki prompt, and aggregate generation throughput is 200+ tok/s at concurrency 4. The user chose 8-bit over 4-bit for quality.