Qwen3.8 27B
on M2 Ultra 128GB · oMLX 0.6.3
Summary
User reports Qwen3.8 27B at 26.4 t/s generation and 276.2 t/s prefill on an Apple Mac Studio M2 Ultra with 128 GB unified memory.
Setup is oMLX 0.6.3 with oQ8e mixed affine 8-bit plus FP16 weights, run as an ANE and CPU split at a source-labelled 65,536-token prompt depth.
The GPU-only control at the same depth is 231.2 prefill and 26.6 decode; the author reports gains apply to cold prefills, not prefix-cached turns, and peak memory at 64K rises from 44.0 GB to 60.0 GB with ANE.