Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3 4B 4-bit at 116.9 t/s generation and 942.3 t/s prompt processing on an M1 Ultra 128GB Mac Studio.
Setup is mlxcel 0.7.0-beta.1 with FP16 KV cache, one request of 512 input tokens and 128 output tokens, greedy temperature 0, no speculative decoding.
The decode figure covers 128 output tokens over 127 forward steps; prefill excludes loading, tokenization and transport, so it is not end-to-end TTFT.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 12 t/s on an M1 Max 64GB.
Setup is Ollama with Q4_K_M and a 65,536-token context, one parallel request.
The 12.0 tok/s is the second 6,872-token prompt run, not an aggregate; separate retrieval tests report prefill of roughly 90 tok/s at about 1,900 and 7,400 tokens, about 83 at 22,200, and 71-74 at 55,600.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3-Coder 30B-A3B at 64.4 t/s generation and 645 t/s prompt processing on an M1 Max 64GB.
Setup is llama.cpp with Q4_K_M, measured via llama-bench on Metal.
The figure is a native prefill/decode rate, not a TTFT-derived estimate; exact test lengths, context, batch settings and KV precision are not given.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports CyberTiel-Coder-35B-A3B (an Ornith 1.5 35B-A3B derivative) at 111.8 t/s generation on a Mac Studio M1 Ultra 64GB.
Setup is oMLX 0.7.0 with the oQ6e mixed affine + FP16 quant, Lightning MTP on, one request, 131K configured context, the whole model in unified memory.
HISTORICAL RESULT, NOT A CURRENT SPEED GUARANTEE. Source-labelled October 7, 2026 CyberTiel FP16 ABBA session: 111.75 tokens/s pooled median across all 18 FP16 B1/B2 requests (display rounded to 111.8). The same author's later October 10 non-contaminated chat-window medians were LOWER: 98.1, 95.0, 91.2 and 94.6 tokens/s, including an after-restart test. One additional 95.0 window was explicitly contaminated by foreign model loads and is excluded. The author suspects background CPU-side jitter but did not establish a cause. These figures are session-dependent.
This historical session is selected because it retains individual completion counts, cache counts and server generation rates. The October 10 export contains window summaries rather than individual request counters. No current-performance or best-runtime claim. No prompt-processing headline: the source's 16K synthetic prefill tests are different requests and cannot be paired with these chat decode results.
MODEL: CyberTiel-Coder-35B-A3B, a modified/abliterated Ornith-1.5-35B-A3B-derived coding/vision MoE, not unmodified Ornith. Approximately 35B total / 3B active, 256 experts with 8 selected. Quantization oQ6e mixed affine, with non-quantized tensors converted from BF16 to FP16, plus the author's fixed 42-key fused MTP head. Published config defaults to affine 6-bit/group64 and has per-module 6/8-bit and group64/128 overrides. Quantized U32 tensors are unchanged by the FP16 conversion. Conversion report: 1,569 BF16 tensors cast, maximum absolute value 25.375; 26,103 nonzero values underflowed to zero. This is a modified model, not a lossless conversion of every tensor.
HARDWARE/RUNTIME: One Mac Studio, Apple M1 Ultra, 64 GPU cores, 64GB unified memory, macOS 15.8.1. The 64GB is one shared memory pool, not additive CPU and GPU RAM. CPU core count, power mode/clocks and sampled peak memory for these chat requests are not recorded. oMLX 0.7.0 build 2987; MLX 0.32.2; mlx-lm 0.31.4.dev; mlx-vlm 0.7.1. Exact loaded runtime commit is unknown. Published model settings: configured context 131072, Lightning MTP enabled with default adaptive depth; DFlash, SpecPrefill, ANE prefill and expert offload disabled. KV precision and explicit Flash Attention state are unspecified. About 28GB of model files is an on-disk size, not this test's peak memory.
WORKLOAD: Three fixed text prompts: Chinese TCP explanation, Python LRU-cache module, and a Chinese arithmetic word problem. Two FP16 arms (B1 and B2) in a same-session BF16/FP16 ABBA comparison, each three rounds of all three prompts, 18 sequential single-stream requests total. Temperature 0, thinking disabled by chat_template_kwargs.enable_thinking=false. Each prompt receives a fresh nonce; all 18 saved cached-token counts are zero. A separate up-to-16-output-token warmup is excluded. Output cap 512 is not the actual length of every answer. Actual prompt-token counts were not saved; 131072 is the configured limit, not the measured input length.
COUNTERS/STATISTIC: Pooled median of the 18 saved native server usage.generation_tokens_per_second values; not tokens divided by request wall time, not a weighted rate, not a median of the two rounded arm medians. Exact underlying unrounded engine durations were not exported. The script separately records TTFT and client end-to-end time, which are not used for the headline.
B1 output tokens / server tok/s, in recorded order:
512/91.99, 512/116.66, 336/118.81, 512/91.50, 512/101.84, 374/121.19, 512/89.16, 512/109.14, 321/117.77.
B2:
512/97.40, 512/112.31, 338/116.14, 512/95.13, 512/113.24, 249/111.19, 512/91.86, 512/114.53, 325/119.02.
The source's separately rounded B1/B2 medians are 109.1/112.3. A separate nine-request FP16 production block is not included in this ABBA statistic. The raw epoch timestamps for the selected requests span 2026-10-06 17:37:10.714721Z through 17:38:45.573781Z, whereas the source folder/description labels the session October 7. The source timezone is unspecified; both date representations are retained.
MODEL ARTIFACT REFERENCE: https://huggingface.co/YCF-AI/CyberTiel-Coder-35B-MLX
Author-published corresponding FP16 artifact:
https://huggingface.co/YCF-AI/CyberTiel-Coder-35B-MLX/tree/2aa5ea8657ad38f57ee0a47e20fd7477e48d0020
That revision was published October 7 at 15:43Z, after the selected local-conversion test. It is NOT a proven tested checkout. Actual tested Hugging Face revision and full tested weight hashes are UNKNOWN. The author describes this repo as its exact daily build; its October 10 local config integrity check matches the published config SHA256 4042ee9152bab8a5eaaaff541235c2ad4a2e874da50b7c0febbb3f854de804c2. Matching config does not establish complete weight-file identity. Six model shards plus model-mtp.safetensors. Earlier BF16 publication: 32e067dd15c07fbfcf841d53efd011f90c6527b7.
PRIMARY EVIDENCE, attributed to YCF-AI / GitHub user 6667084 (third-party measurements, not our run; source data CC BY 4.0):
https://github.com/6667084/omlx-m1-ultra-tuning-notes
Frozen per-request data, all chat_abba_B1 and chat_abba_B2 rows:
https://github.com/6667084/omlx-m1-ultra-tuning-notes/blob/7f56b4094442f4ee67a3ebdee67ff3a7fc08e110/data/2026-10-07/results.jsonl
Measurement script:
https://github.com/6667084/omlx-m1-ultra-tuning-notes/blob/7f56b4094442f4ee67a3ebdee67ff3a7fc08e110/recipes/bench_chat.py
Later lower-speed windows and contamination labels:
https://github.com/6667084/omlx-m1-ultra-tuning-notes/blob/7f56b4094442f4ee67a3ebdee67ff3a7fc08e110/data/2026-10-10-final-check/cybertiel_dialogue_runs.jsonl
See README section 12.8 and adjacent NOTES.md/file_integrity.txt for the retest and identity limitations.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 69.7 t/s generation and 1172.9 t/s prefill on an M1 Ultra 128GB.
Setup is llama.cpp at Q4_K_M, one request, speculative decoding off.
Fazal Majid's llama-bench run published 2026-04-13. A source-reported measurement, not independently reproduced.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Flash-Next at 1.76 t/s generation on a Mac mini M1 16GB, streaming experts from the internal SSD.
Setup is Slotstream 0.2.17 on MLX Swift with PipeNetwork's MLX 4-bit weights, one request, MTP off, a 512-token context and a 19-token prompt.
The model's 105 GB sits on disk while the process peaks near 5 GB of memory. A second run gave the same speed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 25.11 t/s generation and 153.27 t/s prefill on a Mac Studio M1 Ultra 128GB, with a 512-token prompt and 128 generated tokens.
Setup is mlxcel 0.7.0-beta.1 with the mlx-community 4-bit checkpoint, one request, speculative decoding off.
The figures are one timed pass from the mlxcel benchmark results. The harness counts the first token in prefill, so the decode figure would read about 24.91 t/s over 127 intervals.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports IQ2_XS quantized 7B model at 0.95 t/s on an M1 Max 64GB, with 25 tokens generated in 26.30 seconds at context 1523.
Setup is llama.cpp with IQ2_XS quantization; the user also mentions running Mixtral 8x7B fine-tunes at Q5_K_M getting 8-15 t/s.
User also reports an IQ2_XS 120B model taking about 9 minutes for prompt evaluation and roughly 0.1 t/s generation, and questions whether the slow IQ quant performance on Apple Silicon is expected or a bug.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvisionmath
User reports Qwen3.8-27B at 47-48 tok/s on an M1 Ultra 64GB.
Setup is oMLX 0.7.0 with oQ4e quant and MTP fixed depth 2.
MTP off gives 27.4 tok/s, adaptive depth 44.3 tok/s, and DFlash2 23.6 tok/s. The model is also used for vision, scoring 35/35 on a 35-item suite.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen2.5-0.5B Q4 at 95.6 t/s decode with 18 ms TTFT on an Apple M1 16GB, averaged across defaults and normalized modes on the chat_turn workload.
Setup is llama.cpp (llama-server) with Q4 GGUF weights, Metal backend, 1 warmup plus 4 measured reps per cell.
LlamaStash leads raw llama-server (91.9 t/s / 20 ms), LM Studio (88.4 / 68) and Ollama 0.24.0 (79.6 / 102); in defaults mode LlamaStash reaches 99.0 t/s / 15 ms vs raw llama-server's 92.3 / 19, and normalized mode collapses the gap to under 1% (92.2 vs 91.5).
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-contextsummarizationcoding
User benchmarks Qwen3.8-27B at 33.1 tok/s decode on an M1 Max 64GB, comparing Splash, MTPLX M1 fork, MTPLX upstream, oMLX, and TensorFold across 2K to 256K context lengths.
Setup is Splash 1.1.0-m1 with 4-bit group-64 MLX weights and 8-bit KV cache, speculative decoding enabled, single stream. Splash decodes fastest up to 64K (33.1 tok/s at 2K, 24.1 at 64K) and uses the least memory (26.2 GB peak at 128K).
The MTPLX M1 fork had the shortest whole 3-turn conversation at every length (15% shorter than Splash at 128K) but peaked at 43.5 GB. Upstream MTPLX drops to 5.0 tok/s at 128K. The user forked MTPLX and wrote the benchmark.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentictool-use
User compares LFM2.5 2.6B and MiniCPM5 2B on small agentic tool-use tasks on an M1 Air.
LFM2.5 2.6B runs at 200 t/s prompt and 22 t/s generation with llama.cpp, QAD-Q4_0 GGUF, 32k context, using about 2.5 GB of RAM. MiniCPM5 2B runs at 162 t/s prompt and 16 t/s generation with a DSpark draft model, Q8_0 GGUF, 32k context, q5_1 KV cache, using about 3.8 GB of RAM.
User calls LFM2.5 the clear winner: faster, less RAM despite more parameters, and better aligned for English conversation. MiniCPM5 is noted as often responding in Chinese and hallucinating less, but LFM2.5 tends to hallucinate as conversations get longer.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentictool-uselong-context
User reports Qwen3.6-35B-A3B at 25.4 tok/s decode and 38.6 tok/s prefill on a MacBook Pro M1 Pro 32GB.
Setup is llama.cpp build b9466 with Q4_K_S GGUF and q8_0/q4_0 KV cache at 32K context, single sequence, all layers on the Metal GPU.
User compares the MoE against dense Qwen3.6-27B on the same host: the dense model decodes at 5.3 tok/s and prefills at 12.9 tok/s, uses 17.9 GB wired versus 22.4 GB, and swaps ~1 GB versus ~7.6 GB. Prefill is the bottleneck; cold TTFT on a 7.5K prompt is 92.8 s versus 0.2 s warm. Tool-calling was clean in 10/10 episodes.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticsummarization
User reports Gemma 4 E4B at around 8.2 tok/s on an M1 MacBook Air with 16GB unified memory.
Setup is Unsloth Studio with a 4-bit MLX model and a 131.1k context window.
The user says this speed is fine for chatbot use and asks for suggestions of 7B or 8B MLX models for coding and background tasks.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodinglong-context
User reports Qwen3.8-Flash-Next Q2_0 at 44.0 tok/s decode and 328 tok/s prefill at 4K context on a 2021 M1 Max with 64 GB unified memory.
Setup is antirez's ds4 engine (forked for Apple Silicon) with Q2_0 GGUF weights, MTP speculative decoding on, single request, model fully in unified memory. Decode holds 38.8 tok/s at 256K and 35.4 tok/s at 398K; prefill falls from 328 to 292 tok/s over the same range. IQ3_XXS (44 GiB) gives 35.2 tok/s at 4K and 31.7 at 259K.
Sustained five-prompt loop for 5 minutes: 43.2 tok/s first quarter, 43.1 last, 73.1 C, 0.70 J per token. In OpenCode sessions (162 requests, context up to 371K) median decode is 44.7 tok/s under 64K and 38.1 at 128-192K. User compares against a Splash port of Qwen3.8-27B (14 tok/s at 128K+ vs 38) and a DGX Spark running vLLM NVFP4 + MTP (36.8 tok/s prose, 45.8 code). DeepSeek V4 Flash (81 GiB Q2, experts streamed from SSD) runs at 11.8 tok/s at 4K on the same Mac.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM-5.3-Flash at ~0.085 tok/s decode (~11-12 s/token) on a MacBook Pro M1 Pro 32 GB.
Setup is the apus engine with the official FP8-E4M3 checkpoint, 2 GiB expert cache, --tiered streaming of routed experts from NVMe, MTP speculative decoding on by default (draft depth 3).
Prefill was ~76 s for a 26-token prompt, model load ~7 s, peak footprint ~22.3 GiB, weights ~306 GiB, context up to 1,048,576 tokens. The engine reads ~7.9 GiB of experts per generated token cold; the author notes the wall at 32 GB is memory capacity, not disk.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingmath
User reports Qwen3.8-Flash-Next (125B) at 12-15 decode tokens per second on a 2021 M1 Max with 32 GB of unified memory.
Setup is MoEspresso 3 with IQ2_K routed projections and KVarN K4/V4 KV cache, using Metal execution through the mlx-iqk library. Cache-Prior expert bias is enabled by default; prefill is unbiased. The user notes the speed holds only when no other memory-hungry applications are running.
The user also reports an overall score of 84.3% on a reproducible 48-question benchmark, ahead of Claude Opus 4.8 xhigh (80.3%) and other hosted solutions.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcodinglong-context
User reports Qwen3.6-35B-A3B at 144 tok/s on an M1 Max 64GB, up from 99 tok/s with the previous Splash build.
Setup is Inco Splash 1.0.2-m1.1 with 4-bit weights, an 8-bit KV cache and DFlash speculative decoding with 8 draft tokens; time to first token for a 38K prompt is 72 s, down from 106 s.
In a head-to-head on the same Mac with Qwen3.8-27B, Splash decodes at 41.5 tok/s versus 25.9 for MTPLX and 19.8 for oMLX, at 75.3 C and 1.06 J per token; prefill at 44K tokens is 102 tok/s against 125-127 tok/s for the MLX engines.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Ornith (1.0 or 1.5) at 35-60 t/s generation and 500-1100 t/s prefill on an M1 Ultra 64GB, reaching 70 t/s with MTP.
Setup is oMLX with oQ8e quant, 131K context, ANE disabled, native MTP enabled, fp16 models, and no KV cache quantization.
User also tested Qwen 3.8 27B, which starts at 28-30 t/s but drops to 10-15 t/s with prefill of 100-190 t/s, making it unusable for long coding tasks. User asks whether Qwen 3.8 Flash Next is usable on 64GB Macs and which engine to use.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingmath
User reports Qwen3.8 27B at 32.8 t/s average on an M1 Max 64GB MacBook Pro, using the Splash-M1 engine with a 4-bit MLX quant.
Setup is the prebuilt splash-m1 1.0.2-m1 with incoai Splash packages, compared against oMLX 0.6.4. Prefill on an 8K prompt reached 101 t/s. Four parallel requests totalled 62.3 t/s.
Splash was about 2x faster than the 4-bit MLX quant on code, math and parallel requests, but slower on prose (18.3 vs 16.1 t/s) and 32K prompts. Draft acceptance was about 94% on code but only 19% on prose. Time to first token at 32K was about 400 s. Accuracy matched the 4-bit quant on the user's test suites.