M3 Max 96GB reports

generation:

7.4 tokens/s

prompt processing (prefill):

121.3 tokens/s

quant:

8bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6 27B at 7.4 t/s generation and 121.3 t/s prompt processing on an M3 Max 96GB. Setup is oMLX with the MLX 8-bit model at pp1024/tg128, using 28.34 GB peak memory. A pp4096/tg128 run reached 8.8 t/s generation and 133.8 t/s prompt processing. Continuous batching at 4x reached 19.9 t/s aggregate generation. User is new to LLMs and asks whether the slow speed is due to the dense model or a setup problem.

generation:

12.7 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports DeepSeek V4 Flash at 11-13 t/s on an M3 Max 96GB with SSD streaming and iogpu.wired_limit_mb=86016. Setup is antirez's ds4 engine with GGUF. TTFT is 3-5s after warmup. Prefill of 36k tokens takes about 2.5 minutes.