M1 Max 64GB reports

generation:

39.1 tokens/s

prompt processing (prefill):

142.0 tokens/s

quant:

4-bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 39.1 t/s average on an M1 Max 64GB, up from 18.9 t/s with stock Splash kernels. Setup is a ported Inco Splash engine with custom Metal kernels for the M1, 4-bit weights, DFlash speculative decoding, and a 256K context. The 39.1 t/s is the average of npanj's five-prompt Splash benchmark at 250 tokens with default reasoning. A real coding-agent session (opencode, 30K-89K context) ran at 19.5-36.5 t/s decode. Prompt processing was 142 t/s at 2K, 133 t/s at 8K, and 110 t/s at 32K. Four parallel requests reached 65 t/s total. The 35B MoE variant averaged 94.7 t/s with ~450 t/s prefill.

generation:

8.0 tokens/s

prompt processing (prefill):

30.0 tokens/s

quant:

IQ3-XXS

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek-V4-Flash-0731 at about 8 t/s decode and about 30 t/s prefill on an M1 Max 64GB. Setup is a patched llama.cpp with the IQ3-XXS quant at 104 GB and context limited to 64k.

generation:

10.0 tokens/s

prompt processing (prefill):

165.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a run on a 64GB M1 Max. No drafter was used. User mentions n-gram and stripping embeddings as potential optimizations.

generation:

21.0 tokens/s

prompt processing (prefill):

83.0 tokens/s

quant:

Q4 (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a roughly 2x speed boost for Qwen3.8 27B on Apple Silicon using the MTPLX framework. User also reports Qwen3.6 35B A3B at about 55 t/s decode and about 300 t/s prefill, with a peak of 623 t/s.

prompt processing (prefill):

180.0 tokens/s

quant:

Q4 (GGUF)

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports a custom Q4 quant with spliced tensors from Unsloth and AtomicChat quants, with MTP enabled giving +70% decode at 22 btps. Setup uses custom metal-optimized sparse attention, with prefill reduced to 170 tps at 4K and 150 tps at 256K context. Q4_0 MTP matches Q8_0 acceptance rates at half RAM.