Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at roughly 100 tok/s (90-110 through the server) on an M5 Max 128GB with the new ijah engine.
Setup uses EXL3 trellis weights at 5b-6/8b, about 6% behind oMLX but with 5-bit experts.
User says the quant is about 4-5x closer to bf16 than Jundot's oQ4e (KLD 0.033/0.009/0.015 vs 0.128/0.036/0.078) and about 2x closer than Sushi-4bpw. With Inco's DFlash2 drafter the user reports about 120 tok/s but has not verified it yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 50.3 t/s generation and 1084 t/s prompt processing.
Source evidence (verbatim):
Source: https://omlx.ai/benchmarks/performance/ogokptif
# Qwen3.8-27B-oQ5e-mtp on M5 Max (40c) - oMLX Benchmark
Hardware: Apple M5 Max (40-core GPU), 128 GB unified memory. Engine: oMLX v0.7.1.dev1. Model: Qwen3.8-27B-oQ5e-mtp, 5-bit. Multi-token prediction (MTP) is enabled with adaptive max depth 3, so this is a speculative decoding run; DFlash and SpecPrefill are disabled. Single request. At 8k context: 1,084 tokens/s prompt processing and 50.3 tokens/s token generation. At 4k context: 1,137 prompt and 58.4 generation; at 1k: 895.8 prompt and 54.6 generation; at 16k: 882.8 prompt and 75.5 generation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B at 154 t/s decode on a 1k prompt with oMLX's native engine on an M5 Max 128GB, falling to 55 t/s at 262k context.
Setup is oMLX 0.7.1rc1 with the oQ5e-mtp quant, Lightning MTP on, TurboQuant KV off, and a 128-token greedy generation at each context step. Splash 1.3.2 with int8 KV decoded 147 t/s at 1k and 78 t/s at 262k, and used 30.2 GiB against native's 33.5 GiB prefill peak and 37.4 GiB decode.
TurboQuant 5- and 6-bit decoded at 27-29 t/s at 262k, about half of off. The user also ran Qwen3.8-27B, where Splash int8 was faster at almost every context (69 vs 48 t/s at 1k) and native TurboQuant cost 7-25% decode speed with MTP on. No setting showed a detectable accuracy loss of more than about 2 points.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running DeepSeek v3.1 671B Terminus at 3.6 tokens/sec on a MacBook Pro with M5 Max 128GB and 8TB storage.
Setup uses a heavily modified slipstream llama.cpp base that caches MoE experts to disk and calls them in as they change, with a -0.1 bias in the MTP router for in-RAM experts. A custom GGUF-derived format uses BF16 for fixed layers and INT4 for expert layers.
The user says it is working extremely well and generating a somewhat usable 3.6 tokens/sec.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B at 198.6 tok/s generation and 3657.8 tok/s prompt processing on an M5 Max 128GB.
Setup is oMLX 0.7.1dev2 with oQ5e quant, Lightning MTP speculative decoding, and TurboQuant off, at 1024-token prompt and 128-token generation.
Splash engine reaches 182.0 tok/s generation and 6040.8 tok/s prompt processing on the same test; continuous batching at 4x reaches 275.2 tok/s aggregate generation. A second model, Qwen3.7-27B, is also benchmarked.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports GLM 5.3 Flash at 76 tok/s on a 256GB M5 Ultra Mac Studio, run locally with oMLX.
Setup is oMLX on the Mac Studio; the user also runs Qwen 3.8 27B locally for other tasks.
The user says the biggest weakness of local LLMs is being able to run only one instance of GLM, and that coding is a no-go locally, though images and video are fine. They also compare against Ollama Cloud's $20 plan and use Kimi K3 and DeepSeek 4.1 alongside GLM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 79.1 t/s generation.
Source evidence (verbatim):
Source: https://bestllmfor.com/guides/macbook-pro-m5-max-128gb-ai-benchmark/
# MacBook Pro M5 Max 128GB AI benchmark
Joel Ramat ran these tests on his own 16-inch MacBook Pro on September 16, 2026. M5 Max, 18 CPU cores, 40 GPU cores, 128 GB unified memory advertised at 614 GB/s, macOS 27.0, Ollama 0.34.1, plugged in, default GPU memory limit. All figures are raw outputs from Ollama. Each model was run twice with the same prompt (a 300 word explanation of unified memory versus VRAM) and only the second pass kept, one model loaded at a time. Generation speed: gpt-oss 120B MXFP4 (65 GB) 79.1 tok/s; gpt-oss 20B MXFP4 (13 GB) 113.4 tok/s; Qwen 3.6 35B-A3B MLX (23 GB) 167.2 tok/s; Qwen 3.8 27B MLX (18 GB) 68.0 tok/s; Qwen 3.8 27B GGUF Q4 (17 GB) 36.6 tok/s; Gemma 4 12B GGUF Q4 (7.6 GB) 58.1 tok/s. Prefill on a 16,689 token document with gpt-oss 120B: 1,388 tok/s, time to first token 12.0 s, generation 67.6 tok/s on that run. Only short bursts were tested, one machine, one operator.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports running Qwen3.8 27B (4-bit, ~16GB) in LM Studio on a MacBook Pro M5 Max with 48GB unified memory, at around 10-15 tokens/sec.
Setup is LM Studio serving an OpenAI-compatible API to the Pi Coding Agent, which built a React + TypeScript dashboard from scratch over a few hours.
The user says local coding agents are capable but not a drop-in replacement for Claude Code or Codex with frontier models, citing slower inference and reliability concerns.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-context
User reports GLM-5.3-Flash 4-bit with MTP at 36.8 tok/s decode on a disaggregated setup pairing an M5 Ultra 256GB (prefill) with an M3 Ultra 512GB (decode).
Setup is MLX with the JACCL RDMA backend over two Thunderbolt 5 cables, mlx-vlm 0.7.6, a 17.5k-token prompt and 256 output tokens. Prefill runs on the M5 at 988 tok/s (17.8 s), the 358 MB cache is handed off over TB5 in 38 ms, and first token arrives on the M3 about 18.5 s after the request instead of about 40 s.
Single-node figures: M3 Ultra 512GB prefill 438 tok/s at 2k chunks and 454 tok/s at 8k chunks, decode 32.9 tok/s; M5 Ultra 256GB prefill 1,046 tok/s at 2k chunks and 1,188 tok/s at 8k chunks, decode 47.2 tok/s. The user notes the M5 alone decodes faster for a single request, and that the prefill node needs the full model so this only works for models that fit in 256 GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 124.5 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory.
Setup is LM Studio's MLX backend with MLX 4-bit weights; configured context capacity, KV precision and Flash Attention are unknown.
The figure is the article's rounded single-stream decode rate, measured as client-observed streaming decode-phase throughput rather than engine-native timing, and a separate four-stream run reached 265.6 t/s aggregate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 145.8 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory.
Setup is oMLX 0.7.0 with MLX 4-bit weights and an SSD cache directory enabled; KV precision, context capacity and speculative decoding are unknown.
The 145.8 t/s is a rounded single-stream decode rate from a developer comparison, not engine-native timing; a separate four-stream run reached 281.9 t/s aggregate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 173.4 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory.
Setup is Rapid-MLX 0.16.1 default with MLX 4-bit weights and MTP speculative decoding on the text-only lane; configured context capacity, KV precision and Flash Attention are unknown.
The figure is the article's rounded single-stream decode rate, client-observed streaming decode-phase throughput rather than engine-native timing, and the source warns differences under about 5% may be run variation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5 9B at 81.1 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory.
Setup is LM Studio llmster 0.0.25 with the MLX runtime 1.11.0, MLX 4-bit weights, and unknown context capacity, KV precision and Flash Attention.
The figure is the source's rounded single-stream decode rate, client-observed streaming decode-phase throughput rather than engine-native timing; a separate four-stream aggregate result is 230.0 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5 9B at 78 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory.
Setup is oMLX 0.7.0 with MLX 4-bit weights and an SSD cache directory enabled; configured context capacity, KV precision and speculative decoding are unknown.
The figure is the source's rounded single-stream decode rate, client-observed streaming throughput rather than engine-native timing, and the source's homepage and prose give 77.9 t/s instead.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5 9B at 80.1 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory.
Setup is Rapid-MLX 0.16.1 with MLX 4-bit and --no-spec-decode, in image-capable mode benchmarked with text prompts.
The 80.1 t/s is the article's rounded single-stream decode rate, measured as client-observed streaming decode throughput rather than engine-native timing; a separate four-stream run reached 77.0 t/s as an aggregate wall-throughput figure.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.5 9B at 115.5 t/s on a single Mac Studio with an M5 Max and 36 GB of unified memory. Setup is Rapid-MLX 0.16.1 with MLX 4-bit weights and MTP speculative decoding on the text-only lane, with image input disabled. The figure is the article's rounded single-stream decode rate, client-observed streaming decode-phase throughput rather than engine-native timing, and configured context capacity and KV precision are unknown.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 26 t/s on an Apple M5 Max with 36 GB of unified memory.
Setup is LM Studio with MLX 4-bit weights; KV cache and speculative decoding are not established.
The figure is a single-stream decode rate, a median of three requests, not engine-native timing or prompt processing.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 25.3 t/s on an Apple M5 Max with 36 GB unified memory.
Setup is oMLX 0.7.0 with MLX 4-bit weights and an SSD cache directory enabled; speculative decoding is not established.
The figure is a single-stream decode rate from a median of three requests, not engine-native timing or end-to-end throughput, and the source warns differences below roughly 5% can be run variation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 25.8 t/s on an M5 Max with 36 GB of unified memory.
Setup is Rapid-MLX 0.16.1 with MLX 4-bit weights and --no-spec-decode, running text prompts.
The figure is a single-stream streaming decode rate, not engine-native timing or end-to-end throughput, and the harness requests a 512-token cap at temperature 0.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at 37.9 t/s on an Apple M5 Max with 36 GB of unified memory.
Setup is Rapid-MLX 0.16.1 default with MLX 4-bit weights and MTP speculative decoding on the text-only lane, image input disabled.
The figure is a single-stream decode rate, not end-to-end throughput or prompt processing, and comes from a median of three requests with a 512-token cap.