Qwen3.8 27B
M5 Max 128GB · llama.cpp · 262,144 ctx
- generation:
- 31.8 tokens/s
- kv:
- q8_0
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-27B on an M5 Max 128GB across four engines. Setup is llama.cpp with native MTP at 31.8 t/s sustained and 22.7 t/s at 56k context, TTFT 125ms, prefix reuse 0.80, RAM 36.8GB; oMLX 0.6 at 29.8 t/s and 24.6 t/s at depth, TTFT 1968ms, prefix reuse 0.82, ~17GB; mlx-vlm with MTP at 43.3 t/s and 28.9 t/s at depth, TTFT 9982ms, no prefix reuse, 15.6GB; and MTPLX 2.6 at 27.1 t/s and 23.1 t/s at depth, TTFT 13400ms, no prefix reuse. The fastest engine, mlx-vlm, is unusable for agents due to 10s TTFT and no prefix reuse. Without --spec-type draft-mtp, llama.cpp drops the MTP head; a draft cap of 4 gives 31.8 t/s versus 21.9 t/s with no speculation, cap 2 from Qwen3.6 presets gives only 22.2 t/s, and cap 6 loses gains. At 262k context with q8_0 KV cache llama.cpp gives 29.6 t/s and 18.5 t/s at depth; at 131k with f16 it gives 31.8 t/s and 22.7 t/s at depth. The model does 5 searches then calls more tools instead of writing the brief and dies at max_turns. reasoning_effort 'none' throws an exception on 3.8, so enable_thinking: false is needed; oMLX does not activate MTP on 3.8; the model field of the response should be checked rather than /v1/models; and the M5 Max throttles to 50% after 80-110s. No generational verdict versus 3.6 is given.