Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticrp
User reports Swift-1.5-Qwen3.8-27b-oQ8e-mtp at 34.6 tok/s generation and around 400 tok/s prompt processing on an Apple M5 Max 64GB.
Setup is oMLX with oQ8e quant, 262k context window, thinking on at xhigh, MTP speculative decoding enabled.
User compares against base Qwen3.8 27B (32.8 tok/s, ~400 tok/s prompt processing) and notes Swift 1.5 generates fewer tokens per run (51k vs 77k average) with a similar quality score (85.8 vs 85.0).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4-Flash running at ~4.5-5 tok/s decode on an M5 Pro with 48 GB unified memory.
Setup is a from-scratch MLX inference engine that keeps dense/attention weights resident at 8-bit and streams the 256 routed experts (mxfp4) from SSD into a device-side LRU cache, with CAP=1664 and MAX_CTX=32768.
Decode is byte-bound by SSD expert streaming, and prefill and decode both cost about 200 ms per token. Lossless speculative decoding is built but does not fit in 48 GB, projecting ~6-7.5 tok/s at 64 GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at ~26 tok/s generation and ~190 tok/s prompt processing on a MacBook Pro M5 Max with 128 GB unified memory.
Setup is LM Studio with antirez's deepseek_v4 llama.cpp fork sideloaded, IQ2XXS GGUF weights, F16 KV cache, and the full 1048576 token context. The model is a 256-expert MoE with ~80 GB resident.
The figures are approximate and the run is a workaround: upstream llama.cpp does not yet support the deepseek_v4 architecture and LM Studio's MLX runtime rejects V4 GGUFs. Long prompts beyond 16k tokens are flushed in 16k chunks due to the n_batch cap.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 22.99 tokens/s three-scenario decode average on an Apple M5 Pro with 64 GB of unified memory.
Setup is oMLX 0.6.0 with 4-bit affine MLX weights and an external Qwen3.8-27B-MTP-4bit drafter (250.9 MB, block size 3), measured at 16384 tokens of context.
The average combines 26.28 short, 22.81 long, and 19.88 at 11k context. First uncached 11k prefill was 368.32 tokens/s with 30.18 s TTFT. MTP accepted 1,836 of 3,416 drafted tokens (53.7%).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentic
User reports Qwen3.8 27B at 50 t/s on an M5 Pro 64GB.
Setup is Inco Splash with Unsloth UD-Q4_K_XL GGUF weights, run in an agentic setup.
The user credits Splash's optimized kernels, speculative decoding, prefix cache and mixed-weight support, and calls it a breakthrough in local inference on Apple Silicon.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 Flash running at 13-14 tok/s on a MacBook M5 Pro with 48GB.
The model takes 21-24 GB of memory.
The user asks others to share their own Mac specs and preferred local models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-Flash-Next at roughly 18-22 tok/s generation on an M5 Pro MacBook Pro with 48GB unified memory.
Setup is llama.cpp b10930 with Unsloth UD-IQ1_M GGUF, 4096 context, flash attention on, 12 experts on CPU, and --load-mode none --lazy-mode on to avoid Metal OOM.
At 16384 context for an OpenCode test, a 10,634-token prompt took 503 seconds of prefill then generated 192 tokens in 16 seconds; a follow-up reused 10,825 cached tokens but 5,113 more tokens took 165 seconds, and a 28,081-token request was rejected. About 94% of inference time went to prefill, and the user calls it not practical for agentic coding on this setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Inkling-Small 276B-A12B at 2.86 tok/s decode on an M5 with 24GB, with a peak footprint of about 9.5 GB.
Setup is Mference with the pipenetwork/Inkling-Small-MLX-4bit conversion: 3.4 GB resident and about 148 GB on disk.
Decode holds at 2.93 tok/s on a 421-token prompt and 2.56 tok/s on a 2,785-token one, but that longer prefill took 535.9 s. The user also tested an M3 Ultra 256GB at 5.31-6.92 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports GLM-flash-4bit with MTP on an M5 Ultra 512GB, reaching 735.5 prompt t/s and 71.1 generation t/s at 131,072 prompt tokens.
Setup is raw mlx-vlm with --prefill-step-size 8192. At 32,768 tokens the run gave 1056.0 prompt t/s and 72.7 generation t/s; at 65,536 tokens it gave 920.0 prompt t/s and 73.7 generation t/s.
With --prefill-step-size 2048 the same trials gave 623.6 prompt t/s and 50.8 generation t/s at 131,072 tokens. The user also notes omlx with adaptive step size matched 8k performance from 2048 at 64k context and above.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports Qwen3.8 Flash-Next on a base M5 Ultra 96GB (64-core GPU) at a median 828.2 t/s prefill and 42.7 t/s decode per stream, with a single-stream max of 3,328 t/s prefill and 152 t/s decode.
Setup is a custom mlx-serve build with continuous batching at 4-way concurrency, a 4-bit/8-bit mixed MLX quant with MTP, 4 x 128k context (512k total) and 8-bit KV cache using 90GB of unified memory.
Aggregate throughput was ~3,200 t/s prefill and ~170.8 t/s decode at 4-way concurrency. The run covered 112M tokens (109M prompt, 3M generated) with an 89% cache hit rate across ~1,700 sub-agent calls.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a Swift-Qwen3.8-27B 4bit model running on the Splash engine on an M5 Max 128GB, with decode of 79–129 tok/s.
At 64k context the model reaches 87.8 tok/s versus 36.6 tok/s for the same weights in oMLX (oQ4e-mtp, mixed 4/5-bit), and the gap widens with context.
The user notes Splash's /v1 defaults to greedy while Swift wants temp 1.0 / top_k 20 / top_p 0.95, and that the format cannot carry generation_config.json.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Mimo 2.6 Flash at 49.1 t/s generation and 1238.4 t/s prompt processing on an M5 Ultra 256GB, averaged over five trials at 32,000 prompt tokens and 64 generation tokens.
Setup is a custom mlx-vlm patch loading the original weights, with MTP enabled but apparently not working.
A second run at 64,000 prompt tokens and 1024 generation tokens averaged 38.1 t/s generation and 981.9 t/s prompt processing. User also compares Qwen3.8 Flash-Next FP8 on oMLX, reaching 39.7 to 43.9 t/s generation across 32,768 to 200,000 token prompts.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User shares a MacStories review of the M5 Ultra Mac Studio 256GB and a DeepSeek V4.1 Flash comparison against 2x DGX Spark.
Qwen3.8-Flash-Next runs at 108 tok/s on a short prompt and 143 tok/s for code generation on the Mac in MLX oQ4e, versus 42.6 and 39.0 tok/s on 2x DGX Spark in NVFP4 TP2. Prefill is 2,057-2,887 tok/s with TTFT of 24.4 s at 64K, 50.0 s at 128K, and 104 s at 256K.
Decode decays from 90.7 tok/s at 4K to 74.7 tok/s at 256K, and combined throughput rises from 66.2 tok/s at 1 stream to 81.5 tok/s at 3 streams. The comparison is not a controlled A/B: the Mac runs oQ4e on MLX/oMLX while the Sparks run NVFP4/FP8 on vLLM/SGLang.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
mathcodinglong-context
User reports Qwen3.8 27B at 36.9 t/s average on an M5 Pro 64GB using the Splash engine with native 8-bit weights.
Setup is Splash with speculative decoding, 27 GB native 8-bit model, evaluated at temperature 0.0 across five task domains; speeds range from 21.9 to 54.8 t/s.
Context scaling tested to 190,016 tokens with decode speed staying between 21 and 33 t/s; cold prefill at 180k+ tokens takes 4-5 minutes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports Qwen3.8 27B at 144 t/s on an M5 Max MacBook Pro with 36 GB.
Setup is the Inco Splash inference engine, an open-source engine built for Apple silicon, also available as a runtime inside LM Studio. Requirements are M3 or newer, macOS 26.4+, and 36 GB.
The engine is claimed to reach up to 3x the decode speed of Ollama, 2x oMLX, and almost 4x when an agent fans out into sub-agents.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next at ~27.6 t/s generation and ~367 t/s prompt processing on an M5 Pro 64GB, with the 95.5 GiB model streamed from SSD.
Setup is a llama.cpp fork with expert streaming, a draft head, depth 3 with a 0.3 confidence floor, sparse attention, and Metal MoE fusion, using a bartowski Q4_0 checkpoint with Q8_0 output layer and unsloth UD-IQ4_XS spliced in.
Generation drops to 18-18.6 t/s with the draft head off and ~20.6 t/s at 29k context in real chat use. Paired 40-chunk perplexity improved from 5.2777 to 4.3148.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 50 t/s generation and 1800 t/s prompt processing on an M5 Ultra at 8k context.
Figures come from the omlx website rather than a local run, with q4 weights and no MTP. The user notes the benchmarks' provenance is unclear but finds them reasonable.
User calls the results very promising.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at around 20 t/s sustained on an M5 Pro 64GB, with about 30 t/s in the first 2-4k tokens.
User finds it unbearably slow, possibly because the model thinks so much.
User also tried antirez's DS4 with DeepSeek V4 Flash and gets around 8 t/s sustained, which they call slow and basically unusable.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports Nex-N2.5-mini at 133.6 t/s on an Apple M5 Max.
Setup is MLX with the MLX-4bit quant, using temperature 0.7, top_p 0.95, top_k 40, and reasoning_effort high.
User calls the generation speed and prompt processing really good, the memory footprint okay, and the quality good across the board, and plans to use it for agents and possibly coding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks GLM-5.3 321B across a MacBook M5 Max 128GB and an AMD Strix Halo 128GB connected over TB4.
The default tensor split was slower than the Mac alone. A Mac-heavy split improved prompt processing from 188 to 333 t/s and generation from 17 to 24 t/s.
User also reports 12-15 t/s for model sizes of 148-200 GB and plans to add two Asus GX10 next week.