Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports LLaMA-family 7B at 24.1 t/s generation and 221.3 t/s prompt processing on a Mac mini M4 with 32 GB unified RAM.
Setup is llama.cpp build 1550 with Q4_0 and F16 KV cache, Metal backend at ngl 99 on one GPU.
The figures are llama-bench PP512 and TG128 synthetic tests, not a 512-token prompt followed by a 128-token answer; exact source rates are 221.29 and 24.11 t/s.
LLaMA family 7B / Q8_0 / Mac mini M4 Metal b1550 / PP512 TG128 / exact base and artifact unknown
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports LLaMA-family 7B mostly Q8_0 at 13.5 t/s generation and 223.6 t/s prompt processing on a Mac mini M4 with 32 GB unified RAM.
Setup is llama.cpp build 1550 with Q8_0 and F16 KV cache, Metal backend at ngl 99, one GPU.
The figures are llama-bench PP512 and TG128, independent synthetic prompt and generation tests rather than a 512-token prompt followed by a 128-token answer.
LLaMA family 7B / F16 / Mac mini M4 Metal b1550 / PP512 TG128 / exact base and artifact unknown
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a LLaMA-family 7B model at 7.4 t/s generation and 230.2 t/s prompt processing on a Mac mini M4 with 32 GB unified RAM.
Setup is llama.cpp build 1550 with F16 quantization and F16 KV cache, Metal backend at ngl 99, one GPU.
The figures are llama-bench PP512 and TG128 results, independent synthetic prompt and generation tests rather than a 512-token prompt followed by a 128-token answer; exact source rates are 230.18 ± 0.16 t/s and 7.43 ± 0.02 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3 8B at 19.7 t/s generation and 223.8 t/s prompt processing on a Mac mini M4 with 32 GB unified memory.
Setup is llama.cpp b10929 with the Q4_K_M GGUF and Metal, all layers on the GPU, four CPU threads.
The figures are separate llama-bench tests (pp512 and tg128), each the mean of three repetitions at zero initial depth, not one 512-input/128-output request; at 8192 prefill depth generation is 16.1 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3 4B at 36.2 t/s generation and 416.6 t/s prompt processing on a Mac mini M4 with 32 GB unified memory.
Setup is llama.cpp b10929 with the Q4_K_M GGUF and Metal, all layers on the 10-core integrated GPU.
The figures are separate llama-bench tests, pp512 and tg128, each the mean of three repetitions at zero initial depth, not one 512-input/128-output request.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 31B at 5.3 t/s generation and 90 t/s prompt processing on an M4 32GB Mac mini.
Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, 100% GPU offload, 20.29 GB loaded memory including KV cache.
The figures are three-run averages from a fixed Chinese long-form prompt; 4096 is the configured context, not measured input length, and per-run counters are unpublished.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 26B-A4B at 32 t/s generation and 657 t/s prompt processing on an M4 32GB Mac mini.
Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, 100% GPU offload, 17.37 GB loaded memory including KV cache.
Both figures are three-run averages; the 400-character prompt instruction is not an output-token count, and 4096 is the configured context, not measured input length.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 12B at 12.5 t/s generation and 236 t/s prompt processing on an Apple Mac mini M4 with 32 GB unified memory.
Setup is Ollama 0.30.5 with GGUF Q4_K_M and num_ctx=4096, reporting 100% GPU offload and 8.05 GB loaded memory including KV cache.
Both figures are three-run averages; the 4096 is the configured context, not measured input length, and the prompt asks for at least 400 Chinese characters.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 29.9 t/s generation and 566 t/s prompt processing on an Apple Mac mini M4 with 32 GB unified memory.
Setup is Ollama 0.30.5 with GGUF Q4_K_M at num_ctx=4096, with 100% GPU offload and 9.55 GB loaded memory including KV cache.
The figures are source-reported three-run averages, with decode from eval_count/eval_duration and prompt processing from prompt_eval_count/prompt_eval_duration, excluding client and model-loading overhead.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E2B at 55 t/s generation and 1089 t/s prompt processing on an Apple Mac mini M4 with 32 GB unified memory.
Setup is Ollama 0.30.5 with GGUF Q4_K_M at num_ctx=4096 and 100% GPU offload, using the Ollama library gemma4:e2b tag.
The figures are a three-run average (54.97 t/s decode, 1089 t/s prompt) from a fixed Chinese long-form prompt; reported loaded memory is 7.06 GB including KV cache.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 12B at 8.7 t/s generation and 110.4 t/s prompt processing on an M4 32GB Mac mini.
Setup is Ollama 0.30.5 with Q4_K_M and q8_0 KV cache, num_ctx=8192, flash attention enabled, one card.
The row is the first long-context tier: 5713 actual input tokens and 512 actual output tokens, not a full 8192-token prompt or an average. The source TTFT estimate is 64.5 s and request wall time 123.4 s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Gemma 4 E4B at 16.7 t/s generation and 245.5 t/s prompt processing on an M4 32GB Mac mini.
Setup is Ollama 0.30.5 with Q4_K_M and q8_0 KV cache at 8192 context, one Mac mini M4 with 32 GB unified memory.
The figure is a single long-context row with 5709 actual input and 505 actual output tokens, not an average or a full 8192-token prompt; the source TTFT estimate is 23.53 s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 8-22 t/s on a 32GB M4 MacBook Air with 21GB of allocations.
Setup is a custom inference engine called Cherenkov that combines predictive expert streaming with optional mixed-precision execution, keeping a bounded working set of experts in unified memory rather than loading the entire model.
A one-layer lookahead predicts which experts will be needed next and initiates SSD reads.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks about an MTP or DFlash head for Qwen 3.8 27B, and mentions 35B-A3B as a daily driver.
User reports about 8 tok/s for Qwen 3.6 27B with MTP on 32 GB of unified memory.