Qwen3.8 27B
M5 Ultra 96GB · Ollama · 6,600 ctx
- generation:
- 52.9 tokens/s
- prompt processing (prefill):
- 1168.0 tokens/s
- quant:
- MXFP8 (MLX)
- kv:
- q8_0
User reports Qwen3.8-27B at 52.9 t/s decode on a 6.6K prompt and 1,168 t/s prompt eval on a Mac Studio M5 Ultra 96GB (64-core GPU), compared against oMLX at 23.3 t/s cold and 29.7 t/s warm on the same model. Setup is Ollama 0.40.13 with the MLX runner, MXFP8 weights, q8_0 KV cache and a 131072 context; the wired memory limit was raised to 86GB. Other Ollama short-prompt speeds: gpt-oss:120b 95 to 102 t/s, qwen3-coder-next 107 t/s, qwen3.6:35b-a3b 246 t/s, llama4:scout 49 t/s. On long prompts gpt-oss prefill was 2,588 t/s at ~15K and 1,996 t/s at ~60K, Coder-Next 3,382 and 2,687 t/s, Qwen3.8 1,376 and 1,243 t/s. The user expected oMLX to win and it did not, and notes Qwen3.8's thinking volume (10,339 tokens, 161 s) as a real cost.