Local model performance on your hardware

generation:

33.1 tokens/s

quant:

4-bit group-64 (MLX)

kv:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B at 33.1 tok/s decode on an M1 Max 64GB, comparing Splash, MTPLX M1 fork, MTPLX upstream, oMLX, and TensorFold across 2K to 256K context lengths. Setup is Splash 1.1.0-m1 with 4-bit group-64 MLX weights and 8-bit KV cache, speculative decoding enabled, single stream. Splash decodes fastest up to 64K (33.1 tok/s at 2K, 24.1 at 64K) and uses the least memory (26.2 GB peak at 128K). The MTPLX M1 fork had the shortest whole 3-turn conversation at every length (15% shorter than Splash at 128K) but peaked at 43.5 GB. Upstream MTPLX drops to 5.0 tok/s at 128K. The user forked MTPLX and wrote the benchmark.

generation:

144.0 tokens/s

quant:

4-bit

kv:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 144 tok/s on an M1 Max 64GB, up from 99 tok/s with the previous Splash build. Setup is Inco Splash 1.0.2-m1.1 with 4-bit weights, an 8-bit KV cache and DFlash speculative decoding with 8 draft tokens; time to first token for a 38K prompt is 72 s, down from 106 s. In a head-to-head on the same Mac with Qwen3.8-27B, Splash decodes at 41.5 tok/s versus 25.9 for MTPLX and 19.8 for oMLX, at 75.3 C and 1.06 J per token; prefill at 44K tokens is 102 tok/s against 125-127 tok/s for the MLX engines.

generation:

32.8 tokens/s

prompt processing (prefill):

101.0 tokens/s

quant:

4-bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 32.8 t/s average on an M1 Max 64GB MacBook Pro, using the Splash-M1 engine with a 4-bit MLX quant. Setup is the prebuilt splash-m1 1.0.2-m1 with incoai Splash packages, compared against oMLX 0.6.4. Prefill on an 8K prompt reached 101 t/s. Four parallel requests totalled 62.3 t/s. Splash was about 2x faster than the 4-bit MLX quant on code, math and parallel requests, but slower on prose (18.3 vs 16.1 t/s) and 32K prompts. Draft acceptance was about 94% on code but only 19% on prose. Time to first token at 32K was about 400 s. Accuracy matched the 4-bit quant on the user's test suites.

generation:

39.1 tokens/s

prompt processing (prefill):

142.0 tokens/s

quant:

4-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 39.1 t/s average on an M1 Max 64GB, up from 18.9 t/s with stock Splash kernels. Setup is a ported Inco Splash engine with custom Metal kernels for the M1, 4-bit weights, DFlash speculative decoding, and a 256K context. The 39.1 t/s is the average of npanj's five-prompt Splash benchmark at 250 tokens with default reasoning. A real coding-agent session (opencode, 30K-89K context) ran at 19.5-36.5 t/s decode. Prompt processing was 142 t/s at 2K, 133 t/s at 8K, and 110 t/s at 32K. Four parallel requests reached 65 t/s total. The 35B MoE variant averaged 94.7 t/s with ~450 t/s prefill.