Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B at 39.1 t/s average on an M1 Max 64GB, up from 18.9 t/s with stock Splash kernels.
Setup is a ported Inco Splash engine with custom Metal kernels for the M1, 4-bit weights, DFlash speculative decoding, and a 256K context. The 39.1 t/s is the average of npanj's five-prompt Splash benchmark at 250 tokens with default reasoning.
A real coding-agent session (opencode, 30K-89K context) ran at 19.5-36.5 t/s decode. Prompt processing was 142 t/s at 2K, 133 t/s at 8K, and 110 t/s at 32K. Four parallel requests reached 65 t/s total. The 35B MoE variant averaged 94.7 t/s with ~450 t/s prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-usecodinglong-context
User reports Gemma 4 e4b-it-qat at 33 tok/s on a 16 GB M1 Pro, up from 27 tok/s with the non-QAT build.
Setup is Ollama with a 6.4 GiB footprint; the 12B deep model cannot stay co-resident, so each swap costs a 4 to 7 s cold load. Moving per-message content out of the system prompt cut follow-up time to first token from 1.8-2.0 s to 0.4-0.5 s.
Tool-calling accuracy on 9 prompts was 8/9 bare, 5/9 with a 700-token system prompt, and 9/9 with a one-line nudge placed after the question. Memory injection of 109 facts cost 2,300 tokens per turn; recalling the nearest 10 costs about 150 tokens.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares MLX and llama.cpp on Qwen3.8-27B. MLX reaches 15.81 t/s generation and 81.76 t/s prompt processing. llama.cpp reaches 9.69 t/s generation and 99.61 t/s prompt processing.
The llama.cpp run uses the UD-Q4_K_M quant.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a patched engine improved speed from 5-6 to 15-16 tok/s.
Setup uses the Unsloth UD-IQ3_XXS quant.
The user notes a wired limit at 120GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at about 8 t/s decode and about 30 t/s prefill on an M1 Max 64GB.
Setup is a patched llama.cpp with the IQ3-XXS quant at 104 GB and context limited to 64k.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a run on a 64GB M1 Max.
No drafter was used.
User mentions n-gram and stripping embeddings as potential optimizations.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks a custom llama.cpp branch with faster Metal inference and n-gram SSD offload at pp512 530.18 t/s and tg128 31.17 t/s with resident n-gram.
The model is 68.37 GiB and 125.74 B params.
With SSD read mode, pp512 is 197.92 t/s and tg128 is 28.47 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a roughly 2x speed boost for Qwen3.8 27B on Apple Silicon using the MTPLX framework.
User also reports Qwen3.6 35B A3B at about 55 t/s decode and about 300 t/s prefill, with a peak of 623 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a custom Q4 quant with spliced tensors from Unsloth and AtomicChat quants, with MTP enabled giving +70% decode at 22 btps.
Setup uses custom metal-optimized sparse attention, with prefill reduced to 170 tps at 4K and 150 tps at 256K context.
Q4_0 MTP matches Q8_0 acceptance rates at half RAM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
text-generation
User reports Gemma 4 at 15-20 t/s, a range the user calls usable for simple tasks.