Local model performance on your hardware

generation:

32.7 tokens/s

prompt processing (prefill):

420.8 tokens/s

quant:

UD-Q4_K_M (GGUF)

kv:

Q8

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports MiniMax M2.7 at 32.7 t/s generation and 420.8 t/s prefill on an M2 Ultra 192GB. Setup is llama.cpp at UD-Q4_K_M, one request, speculative decoding off. Token-weighted rates across four sequential agentic tasks in crypdick's run of 2026-08-10. A source-reported measurement, not independently reproduced.

generation:

37.9 tokens/s

prompt processing (prefill):

301.6 tokens/s

quant:

UD-Q3_K_M (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports MiMo-V2.5 at 37.87 t/s generation and 301.61 t/s prompt processing on an M2 Ultra 192GB Mac Studio. Setup is llama.cpp llama-server with UD-Q3_K_M GGUF, q8_0 KV cache, 65,536 context, Flash Attention on, gpu_layers 999, batch 2048, and 8,192 MiB cache_ram. Figures are token/time-weighted across four sequential real-world agentic tasks (n=1 each), not llama-bench or end-to-end throughput; one task hit its 8,192-token output cap and 258 of 2,290 prompt tokens were cached. Server peak RSS was 131.015 GiB.

generation:

22.4 tokens/s

prompt processing (prefill):

360.2 tokens/s

quant:

Q6_K_XL (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B at 22.38 t/s generation (tg128) and 360.22 t/s prompt processing (pp512) on a Mac Studio M2 Ultra with 192GB unified memory. Setup is llama.cpp build b10450 llama-bench with the Unsloth UD-Q6_K_XL quant (24.13 GiB), flash attention on, q8_0 K and V cache and batch and micro-batch 2048. The user serves the model through llama-server at 131072 context with MTP speculative decoding (draft-mtp, draft max 3), and gets around 16 t/s on the WebUI with a flappy bird game prompt.

generation:

25.8 tokens/s

prompt processing (prefill):

250.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4 Flash at 25.8 t/s, with a 42 t/s peak, on an M2 Ultra with 60 cores and 192gb. Setup is a custom llama.cpp fork with the model repacked losslessly to 141GiB, smaller than the public GGUFs, no KV cache quant, and an SSD KV cache with dynamic lanes, 1M context total across 8 lanes. Prompt processing is about 250 t/s at 8k-32k. The user compares this with 16 t/s on an M3 Ultra.