Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares MLX and GGUF for Qwen3.8 27B on an M5 Pro, reporting generation at about 19 t/s.
Setup is llama.cpp with Metal, which the user says now matches MLX prefill at roughly 300-350 t/s.
The user questions whether MLX is still needed and mentions MTP and oMLX.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks prompt processing at 65,536 context, with throughput falling from 62.80 t/s at 128 context to 33.92 t/s at 65,536.
Memory peaked at ~120+ GB, and the system stayed sluggish but usable.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-use
User reports Tencent HY3 295B-A21B MoE at 32.4 t/s generation at empty context and 16.3 t/s at 16K context on Metal.
Setup is llama.cpp with the UD128 quant (107 GB), a q8_0 KV cache, and 24K context. With MTP n=2, generation peaks at 38 t/s.
Prefill runs 528 t/s at pp512 empty and 124 t/s at pp512 at 16K. User compares the result favorably to DeepSeek V4 Flash.
User reports a new MLX quantization method, WinterMix, for Qwen3.5-122B-A10B, with two builds: WinterMix58 at 82 GiB and WinterMix48 at 68 GiB.
Benchmarks show perplexity improvements over existing MLX quants.
The user is enthusiastic about the results and the method's advantages for agentic workflows on Apple Silicon.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports custom w8a8 kernels on an M5 MacBook Air, with prefill improving from 2193 t/s to 3029 t/s for 130k tokens.
User also mentions llama.cpp for Macs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Gemma 4 26B at 97 tokens/s in llama.cpp on a MacBook Pro M5 Max.
Setup is GGUF weights, with a prompt asking for a recursive Python Fibonacci program.
With the user's Multi-Token Prediction (MTP) implementation for llama.cpp and quantized Gemma 4 assistant draft models, the same prompt runs at 138 tokens/s, which the user calls 40% faster.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingcreative-writing
User reports 63 t/s on Qwen3.6-27B 4-bit MLX on an M5 Max 64GB with the MTPLX engine, up from a 28 t/s baseline.
Setup is a custom patched MLX fork with Metal kernels, using native MTP heads at temperature 0.6, top_p 0.95 and top_k 20, with optimal depth D3.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-context
User reports 160 t/s prefill and 5-6 t/s generation on an M5 Max 128GB with Qwen 3.6 27B Q8 MLX at 290k context.
GPU utilization sits at only 36-50%.
The user finds the generation rate low against an expected 8-14 t/s and asks others for comparison.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma4-31B at 7.5 t/s and Qwen3.6-27B locking up on an M5 Max with 128 GB, while Qwen3.6-35B-A3 is fast.
Setup uses DFLASH.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen 3.6 27B at 32 t/s on a MacBook Pro M5 Max 64GB, generating 33,946 tokens in 18m04s.
User compares Gemma 4 31B on the same hardware at 27 t/s, generating 6,209 tokens in 3m51s.
User finds Qwen showed more creativity while Gemma won for game logic.