Local model performance on your hardware

generation:

111.8 tokens/s

quant:

oQ6e mixed affine + FP16 (MLX)

mtp (multi-token prediction):

on

User reports CyberTiel-Coder-35B-A3B (an Ornith 1.5 35B-A3B derivative) at 111.8 t/s generation on a Mac Studio M1 Ultra 64GB. Setup is oMLX 0.7.0 with the oQ6e mixed affine + FP16 quant, Lightning MTP on, one request, 131K configured context, the whole model in unified memory. HISTORICAL RESULT, NOT A CURRENT SPEED GUARANTEE. Source-labelled October 7, 2026 CyberTiel FP16 ABBA session: 111.75 tokens/s pooled median across all 18 FP16 B1/B2 requests (display rounded to 111.8). The same author's later October 10 non-contaminated chat-window medians were LOWER: 98.1, 95.0, 91.2 and 94.6 tokens/s, including an after-restart test. One additional 95.0 window was explicitly contaminated by foreign model loads and is excluded. The author suspects background CPU-side jitter but did not establish a cause. These figures are session-dependent. This historical session is selected because it retains individual completion counts, cache counts and server generation rates. The October 10 export contains window summaries rather than individual request counters. No current-performance or best-runtime claim. No prompt-processing headline: the source's 16K synthetic prefill tests are different requests and cannot be paired with these chat decode results. MODEL: CyberTiel-Coder-35B-A3B, a modified/abliterated Ornith-1.5-35B-A3B-derived coding/vision MoE, not unmodified Ornith. Approximately 35B total / 3B active, 256 experts with 8 selected. Quantization oQ6e mixed affine, with non-quantized tensors converted from BF16 to FP16, plus the author's fixed 42-key fused MTP head. Published config defaults to affine 6-bit/group64 and has per-module 6/8-bit and group64/128 overrides. Quantized U32 tensors are unchanged by the FP16 conversion. Conversion report: 1,569 BF16 tensors cast, maximum absolute value 25.375; 26,103 nonzero values underflowed to zero. This is a modified model, not a lossless conversion of every tensor. HARDWARE/RUNTIME: One Mac Studio, Apple M1 Ultra, 64 GPU cores, 64GB unified memory, macOS 15.8.1. The 64GB is one shared memory pool, not additive CPU and GPU RAM. CPU core count, power mode/clocks and sampled peak memory for these chat requests are not recorded. oMLX 0.7.0 build 2987; MLX 0.32.2; mlx-lm 0.31.4.dev; mlx-vlm 0.7.1. Exact loaded runtime commit is unknown. Published model settings: configured context 131072, Lightning MTP enabled with default adaptive depth; DFlash, SpecPrefill, ANE prefill and expert offload disabled. KV precision and explicit Flash Attention state are unspecified. About 28GB of model files is an on-disk size, not this test's peak memory. WORKLOAD: Three fixed text prompts: Chinese TCP explanation, Python LRU-cache module, and a Chinese arithmetic word problem. Two FP16 arms (B1 and B2) in a same-session BF16/FP16 ABBA comparison, each three rounds of all three prompts, 18 sequential single-stream requests total. Temperature 0, thinking disabled by chat_template_kwargs.enable_thinking=false. Each prompt receives a fresh nonce; all 18 saved cached-token counts are zero. A separate up-to-16-output-token warmup is excluded. Output cap 512 is not the actual length of every answer. Actual prompt-token counts were not saved; 131072 is the configured limit, not the measured input length. COUNTERS/STATISTIC: Pooled median of the 18 saved native server usage.generation_tokens_per_second values; not tokens divided by request wall time, not a weighted rate, not a median of the two rounded arm medians. Exact underlying unrounded engine durations were not exported. The script separately records TTFT and client end-to-end time, which are not used for the headline. B1 output tokens / server tok/s, in recorded order: 512/91.99, 512/116.66, 336/118.81, 512/91.50, 512/101.84, 374/121.19, 512/89.16, 512/109.14, 321/117.77. B2: 512/97.40, 512/112.31, 338/116.14, 512/95.13, 512/113.24, 249/111.19, 512/91.86, 512/114.53, 325/119.02. The source's separately rounded B1/B2 medians are 109.1/112.3. A separate nine-request FP16 production block is not included in this ABBA statistic. The raw epoch timestamps for the selected requests span 2026-10-06 17:37:10.714721Z through 17:38:45.573781Z, whereas the source folder/description labels the session October 7. The source timezone is unspecified; both date representations are retained. MODEL ARTIFACT REFERENCE: https://huggingface.co/YCF-AI/CyberTiel-Coder-35B-MLX Author-published corresponding FP16 artifact: https://huggingface.co/YCF-AI/CyberTiel-Coder-35B-MLX/tree/2aa5ea8657ad38f57ee0a47e20fd7477e48d0020 That revision was published October 7 at 15:43Z, after the selected local-conversion test. It is NOT a proven tested checkout. Actual tested Hugging Face revision and full tested weight hashes are UNKNOWN. The author describes this repo as its exact daily build; its October 10 local config integrity check matches the published config SHA256 4042ee9152bab8a5eaaaff541235c2ad4a2e874da50b7c0febbb3f854de804c2. Matching config does not establish complete weight-file identity. Six model shards plus model-mtp.safetensors. Earlier BF16 publication: 32e067dd15c07fbfcf841d53efd011f90c6527b7. PRIMARY EVIDENCE, attributed to YCF-AI / GitHub user 6667084 (third-party measurements, not our run; source data CC BY 4.0): https://github.com/6667084/omlx-m1-ultra-tuning-notes Frozen per-request data, all chat_abba_B1 and chat_abba_B2 rows: https://github.com/6667084/omlx-m1-ultra-tuning-notes/blob/7f56b4094442f4ee67a3ebdee67ff3a7fc08e110/data/2026-10-07/results.jsonl Measurement script: https://github.com/6667084/omlx-m1-ultra-tuning-notes/blob/7f56b4094442f4ee67a3ebdee67ff3a7fc08e110/recipes/bench_chat.py Later lower-speed windows and contamination labels: https://github.com/6667084/omlx-m1-ultra-tuning-notes/blob/7f56b4094442f4ee67a3ebdee67ff3a7fc08e110/data/2026-10-10-final-check/cybertiel_dialogue_runs.jsonl See README section 12.8 and adjacent NOTES.md/file_integrity.txt for the retest and identity limitations.

generation:

47-48 tokens/s

quant:

oQ4e (MLX)

User reports Qwen3.8-27B at 47-48 tok/s on an M1 Ultra 64GB. Setup is oMLX 0.7.0 with oQ4e quant and MTP fixed depth 2. MTP off gives 27.4 tok/s, adaptive depth 44.3 tok/s, and DFlash2 23.6 tok/s. The model is also used for vision, scoring 35/35 on a 35-item suite.

generation:

35-60 tokens/s

prompt processing (prefill):

500-1100 tokens/s

quant:

oQ8e

User reports Ornith (1.0 or 1.5) at 35-60 t/s generation and 500-1100 t/s prefill on an M1 Ultra 64GB, reaching 70 t/s with MTP. Setup is oMLX with oQ8e quant, 131K context, ANE disabled, native MTP enabled, fp16 models, and no KV cache quantization. User also tested Qwen 3.8 27B, which starts at 28-30 t/s but drops to 10-15 t/s with prefill of 100-190 t/s, making it unusable for long coding tasks. User asks whether Qwen 3.8 Flash Next is usable on 64GB Macs and which engine to use.