Local model performance on your hardware

generation:

35.3 tokens/s

prompt processing (prefill):

528.9 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8 27B at 35.28 t/s decode and 528.91 t/s prefill on an M5 Pro 64GB Mac Mini. Setup is Splash with Q8_0 GGUF weights and 4k input prompts. User also compares llama.cpp, MLX, and Splash across several quants, and reports an M1 Max 64GB Splash run at 135.96 t/s decode and 915.01 t/s prefill. User settled on Splash with Q8_0 and is happy with the results.

generation:

50.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 50 t/s on an M5 Pro 64GB. Setup is Inco Splash with Unsloth UD-Q4_K_XL GGUF weights, run in an agentic setup. The user credits Splash's optimized kernels, speculative decoding, prefix cache and mixed-weight support, and calls it a breakthrough in local inference on Apple Silicon.

generation:

36.9 tokens/s

quant:

Q8 (Splash)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 36.9 t/s average on an M5 Pro 64GB using the Splash engine with native 8-bit weights. Setup is Splash with speculative decoding, 27 GB native 8-bit model, evaluated at temperature 0.0 across five task domains; speeds range from 21.9 to 54.8 t/s. Context scaling tested to 190,016 tokens with decode speed staying between 21 and 33 t/s; cold prefill at 180k+ tokens takes 4-5 minutes.