Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM-5.3 Flash at 68.8 tok/s on an M5 Ultra 256GB.
Setup is oMLX 0.7.0 with oQ4e quantization and MTP speculative decoding, with more optimization still to come.
Prefill is 1,878 toks. User shared the result because other benchmarks looked lower than expected.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen 3.8 Flash-Next (125B-A6B MoE) at 154.5 tok/s decode and 4,183 tok/s prefill on a single Mac Studio M5 Ultra 256 GB.
Setup is mlx-serve 26.9.7-dev with the MLX-Serve mixed 4/8-bit pack (100 GiB), KV cache bf16, MTP speculative decoding, PLD and batched decode, at 262,144 context.
Decode holds ~150 tok/s through 66k and is 114.4 tok/s at 256k; prefill stays flat at ~4.1-4.35k tok/s. A 4-stream burst gives 166.8 tok/s aggregate versus 119.1 alone. The user notes long-context recall was not verified because answers never cited the planted value.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User asks whether oMLX benchmarks can be spoofed, citing a third-party oMLX benchmark entry for deepseek-v4.1-flash-q3-g128-lsq-mlx on an M5 Ultra with 256GB RAM, posted two days earlier, reporting about 73.4 tok/s generation and 1,010 tok/s prompt processing.
The run is not the user's own; the figures come from a linked oMLX benchmark page and screenshot.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodinglong-context
User benchmarks Qwen3.8-Flash-Next 8-bit on a Mac Studio M5 Ultra 256GB, reporting 17 t/s decode and 1,069 t/s prefill at 250k context on llama.cpp (Metal).
Setup is llama.cpp with Q8_0 weights and MTP off; oMLX 0.7.0rc1 reaches 51 t/s decode and 1,365 t/s prefill at the same depth, and 88 t/s decode at 10k with MTP on.
A cold 250k-token prompt took 234 s before the first token, while the cached next step took 0.64 s. DeepSeek-V4-Flash at 400k and 500k context took 934 s and 1,311 s cold. The user notes one overnight agent run per setup and that the DeepSeek engine comparison mixes two model variants.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codinglong-context
User reports Qwen3.8-Flash-Next 8-bit at 29 t/s decode on a 1M-token prompt on an M5 Ultra 256GB Mac Studio.
Setup is oMLX 0.7.0 with oQ8e weights and YaRN x4 position scaling; the 1M-token first read took 5.3 minutes and the next turn 6.7 s, finding all three hidden codes.
At 250k context the model prefills at 4,235 t/s and decodes at 63 t/s; the user notes YaRN stretches a 262k-trained model and that a needle test does not prove reasoning at 1M.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User compares Qwen3.8-Flash-Next against Laguna-S-2.1 on a Mac Studio M5 Ultra 256GB, both capped at 262K context with thinking on and unique content per run.
Qwen runs under oMLX with an oQ8e quant and MTP speculation, holding roughly 4,200 tok/s prefill and 59-74 tok/s decode across sizes. Laguna runs under LM Studio with an 8-bit quant, prefill degrading superlinearly from 10.3s at 8K to 455.4s at 200K, and decode falling from 68 to 34 tok/s with no speculation.
Quality was a draw at 4/4 each on four script-verified problems. At 200K, prefill is 94% of total time on both. The user notes an earlier run was invalidated by shared prefixes letting the KV cache carry over.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentic
User reports Qwen3.8-Flash-Next at 108 t/s generation and 2,887 t/s prompt processing on an M5 Ultra 256GB Mac Studio.
Setup is oMLX 0.7.0.dev2 with MLX backend, oQ4e 4-bit quant, multi-token prediction depth 3, 16K prompt.
M3 Ultra 512GB comparison: 70 t/s generation, 1,143 t/s prompt. At 256K context, M5 prompt processing was 2,544 t/s. Qwen3.8-27B on M5 Ultra: 48 t/s generation, 1,701 t/s prompt; RTX 5090 PC in LM Studio: 59 t/s generation, 3,031 t/s prompt.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Mimo 2.6 Flash at 49.1 t/s generation and 1238.4 t/s prompt processing on an M5 Ultra 256GB, averaged over five trials at 32,000 prompt tokens and 64 generation tokens.
Setup is a custom mlx-vlm patch loading the original weights, with MTP enabled but apparently not working.
A second run at 64,000 prompt tokens and 1024 generation tokens averaged 38.1 t/s generation and 981.9 t/s prompt processing. User also compares Qwen3.8 Flash-Next FP8 on oMLX, reaching 39.7 to 43.9 t/s generation across 32,768 to 200,000 token prompts.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User shares a MacStories review of the M5 Ultra Mac Studio 256GB and a DeepSeek V4.1 Flash comparison against 2x DGX Spark.
Qwen3.8-Flash-Next runs at 108 tok/s on a short prompt and 143 tok/s for code generation on the Mac in MLX oQ4e, versus 42.6 and 39.0 tok/s on 2x DGX Spark in NVFP4 TP2. Prefill is 2,057-2,887 tok/s with TTFT of 24.4 s at 64K, 50.0 s at 128K, and 104 s at 256K.
Decode decays from 90.7 tok/s at 4K to 74.7 tok/s at 256K, and combined throughput rises from 66.2 tok/s at 1 stream to 81.5 tok/s at 3 streams. The comparison is not a controlled A/B: the Mac runs oQ4e on MLX/oMLX while the Sparks run NVFP4/FP8 on vLLM/SGLang.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 50 t/s generation and 1800 t/s prompt processing on an M5 Ultra at 8k context.
Figures come from the omlx website rather than a local run, with q4 weights and no MTP. The user notes the benchmarks' provenance is unclear but finds them reasonable.
User calls the results very promising.