Local model performance on your hardware

generation:

90.0 tokens/s

quant:

oQ4 (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6 35B-A3B at 90 tok/s on an M4 Pro Mac mini with 48GB unified memory. Setup is oMLX 0.7.0 with a custom oQ4 quantised model using MTP headers. User notes performance had recently fallen as low as 10-11 tok/s before this improvement.

generation:

24.8 tokens/s

quant:

oQ4e (MLX)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 24.8 tok/s decode on an M4 Pro 48GB, with median TTFT 943 ms over three samples of 4096 prompt tokens and 128 generated tokens. Setup is oMLX 0.6.4 with the oQ4e 4-bit MLX model and Lightning MTP draft depth 3 enabled via the mtp3 profile, at a 200k context window. Enabling MTP through a profile rather than settings.json raised decode from 10 tok/s to 24.8 tok/s. A separate sweep documents the MTP-on prefill curve at 1.8k-30k prompts and a 5-turn agentic session reaching 157k context with 85% token reuse.

generation:

1.8 tokens/s

quant:

JANG

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks MoE streaming in oMLX on an M4 Pro 48GB across four models. GLM-5.3-Flash-JANG-MTP reaches 1.82 tok/s with 12.36s TTFT, 10.52 GiB after load and 14.68 GiB peak. Qwen3.8-JANG 4S reaches 3.63 tok/s with 8.56s TTFT, 7.04 GiB after load and 11.19 GiB peak. Qwen3.8-JANG 4M reaches 3.19 tok/s with 11.23s TTFT, 7.05 GiB after load and 11.11 GiB peak. DeepSeek-V4-Flash-0731-JANG reaches 2.71 tok/s with 6.52s TTFT, 8.25 GiB after load and 16.87 GiB peak. The primary record uses GLM-5.3-Flash-JANG-MTP. The other models are Qwen3.8 in JANG 4S and 4M quants and DeepSeek V4 Flash in the 0731 variant with the JANG quant. MoE streaming allows larger MoE models to run with a lower memory footprint at the cost of speed.