Qwen3.8 27B
on M3 Max 128GB · MTPLX · 1,024 ctx
Use cases
codinglong-context
Summary
User benchmarks Qwen3.8-27B at 41.4 tok/s decode and 198.2 tok/s prefill at 1K context on an Apple M3 Max 128GB (40-core GPU).
Setup is MTPLX v2.8.2 with a 4-bit dynamic/GS32 MLX model, MTP depth 3, single request, 128-token generations, 3-minute GPU rests between rounds.
MTPLX decode falls to 19.8 tok/s and prefill to 115.5 tok/s at 16K, where oMLX oQ4e reaches 27.0 tok/s decode and 231.7 tok/s prefill; the 8-bit MTPLX Quality build is nearly identical in speed (18.8 tok/s, 121.3 tok/s) with KL 0.00105 versus Speed's 0.0220. ANE prefill on oMLX gains 57.0% at 32K (126.1 to 198.1 tok/s).