Qwen3.8 27B
on M3 Ultra 512GB · oMLX · 262,144 ctx
Sep 28, 2026
Use cases
agentictool-uselong-context
Summary
User reports Qwen3.8-27B at about 65-75 tok/s generation at 8K context on one Mac Studio M3 Ultra 512GB (one of four run as independent replicas).
Setup is omlx 0.6.1 with an oQ4e MLX quant, MTP speculative decoding with 3 draft tokens, and a 262144-token context cap. Prefill measured at 420-442 tok/s; batch-8 throughput reached 274 tok/s.
MTP draft depth 3 was the peak (65-75 tok/s vs 50.0 at depth 1 and 65.6 at depth 4). oQ8e was tied on quality but slower at 43 tok/s. A Mac mini M4 Pro 64GB reached 20.8 tok/s and OOM'd around 168k context. An RTX 5090 reference run under llama.cpp with NVFP4+MTP hit ~145 tok/s.