Local model performance on your hardware

generation:

144.4 tokens/s

quant:

MLX 4-bit affine, group 64 (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3 30B-A3B at 144.4 t/s on an M5 Max with 128GB unified memory. Setup is MLX with 4-bit affine quantization, group size 64, standard 8 routed experts per token, batch 1 with a 944-token prompt and 512 output tokens. The figure is the arithmetic mean of six fresh-process runs, ranging 144.10 to 145.01 t/s; an altered 6-expert routing gave 155.61 t/s but is not a standard-model result.

generation:

130.6 tokens/s

prompt processing (prefill):

5402.4 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3 30B-A3B at 130.628 t/s generation and 5402.434 t/s prompt processing on an Apple M5 Max with 128GB unified memory. Setup is MLX 0.32.2 with MLX-LM 0.31.3 on macOS 27.0; quantization, context length and KV cache settings are not reported. Figures come from a third-party hardware review (Tbreak, September 21, 2026), not a reproducible benchmark log; prompt and generated token counts are unknown.

generation:

133.6 tokens/s

quant:

MLX-4bit (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Nex-N2.5-mini at 133.6 t/s on an Apple M5 Max. Setup is MLX with the MLX-4bit quant, using temperature 0.7, top_p 0.95, top_k 40, and reasoning_effort high. User calls the generation speed and prompt processing really good, the memory footprint okay, and the quality good across the board, and plans to use it for agents and possibly coding.

generation:

17.2 tokens/s

prompt processing (prefill):

706.0 tokens/s

quant:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks a 128 GB M5 Max MacBook Pro across multiple models, selecting Qwen 3.6 27B MLX Q8 as the primary one. Other models tested include Gemma 4, MiniMax M2.7 and DeepSeek V4 Flash. The user is enthusiastic about local AI.

generation:

5-6 tokens/s

prompt processing (prefill):

160.0 tokens/s

quant:

Q8 (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 160 t/s prefill and 5-6 t/s generation on an M5 Max 128GB with Qwen 3.6 27B Q8 MLX at 290k context. GPU utilization sits at only 36-50%. The user finds the generation rate low against an expected 8-14 t/s and asks others for comparison.