Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports the Flint model at around 112 tok/s on an M5 Pro 48GB Mac.
Setup is Ollama, with the speed read from Ollama's own eval_count / eval_duration and the model unloaded after each run.
The figure comes from the author's own speed-test tool, which streams the answer and counts reasoning tokens toward the speed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B running at 10-20 tok/s on an M5 Pro Mac Mini with 48GB RAM, below the 30-50 tok/s they expected from community optimizations.
Setup is oMLX 0.70 with both 4-bit and 8-bit models; the 4-bit run took 40 minutes on a lengthy command and the 8-bit run progressed only 5% in over 10 minutes before being cancelled.
The user asks whether this performance is expected for their hardware.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3-Coder-Next 80B-A3B at 34.5 tok/s decode on an M5 Pro 48GB Mac, running a full agentic coding task in 44s.
Setup is the Global Agent engine (MLX-based) with 6-bit weights and expert streaming from SSD, speed mode, peak memory 31.5GB.
The post also benchmarks Qwen3.8-Flash-Next 229B-A7B (19.2 tok/s), Qwen3-235B-A22B (9.19 tok/s) and MiniMax-M3 426B-A24B (4.62 tok/s) on the same machine, plus 16GB and 32GB memory-capped runs; the 16GB Qwen3-Coder-Next run reached 41.2 tok/s. Exact mode figures are lower (17.7, 7.88, 2.62, 2.20 tok/s).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.5-27B affine 4bit prefilling a 4096-token prompt at 461 tok/s on an M5 Pro 48GB (20-core GPU) with stock mlx 0.32.0 and mlx_lm 0.31.3 in a clean venv.
The post is a bug report against oMLX 0.5.0, whose bundled MLX build fails a NAX metallib scan, so its M5 dispatch never engages: the same model prefilled at only 180 tok/s at 4096 tokens there, and 262 tok/s with OMLX_NAX=1. No decode figure is given beyond saying decode is normal.
Qwen3.5-35B-A3B 4bit prefilled at 1,798 tok/s in the clean venv against about 900 on oMLX 0.5.0.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4-Flash running at ~4.5-5 tok/s decode on an M5 Pro with 48 GB unified memory.
Setup is a from-scratch MLX inference engine that keeps dense/attention weights resident at 8-bit and streams the 256 routed experts (mxfp4) from SSD into a device-side LRU cache, with CAP=1664 and MAX_CTX=32768.
Decode is byte-bound by SSD expert streaming, and prefill and decode both cost about 200 ms per token. Lossless speculative decoding is built but does not fit in 48 GB, projecting ~6-7.5 tok/s at 64 GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 Flash running at 13-14 tok/s on a MacBook M5 Pro with 48GB.
The model takes 21-24 GB of memory.
The user asks others to share their own Mac specs and preferred local models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-Flash-Next at roughly 18-22 tok/s generation on an M5 Pro MacBook Pro with 48GB unified memory.
Setup is llama.cpp b10930 with Unsloth UD-IQ1_M GGUF, 4096 context, flash attention on, 12 experts on CPU, and --load-mode none --lazy-mode on to avoid Metal OOM.
At 16384 context for an OpenCode test, a 10,634-token prompt took 503 seconds of prefill then generated 192 tokens in 16 seconds; a follow-up reused 10,825 cached tokens but 5,113 more tokens took 165 seconds, and a 28,081-token request was rejected. About 94% of inference time went to prefill, and the user calls it not practical for agentic coding on this setup.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-usemath
User compares DeepSeek V4 Flash 9B distill against its base model Qwen3.5 9B on a MacBook Pro M5 Pro with 48 GB RAM.
Both models gave the same answers on 6/8 tasks, but the distill used fewer tokens (5480 vs 8975). Throughput was near identical (44 vs 41 tok/s).
The distill was slower on open-ended tasks (1467 tokens vs 867 for a buffer overflow explanation). The base model failed arithmetic due to a token cap.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares MLX and GGUF for Qwen3.8 27B on an M5 Pro, reporting generation at about 19 t/s.
Setup is llama.cpp with Metal, which the user says now matches MLX prefill at roughly 300-350 t/s.
The user questions whether MLX is still needed and mentions MTP and oMLX.