Qwen3.8 27B
on M2 Ultra 128GB · oMLX
Use cases
summarizationagenticlong-context
Summary
User reports Qwen3.8 27B at 23 t/s generation and 304.2 t/s prompt processing on an M2 Ultra 128GB Mac Studio.
Setup is oMLX 0.6.3 with oQ4e 4-bit quantization at workload depth 65536, with 16.7GiB of weights and peak memory rising from 25.3 to 36.2GB.
The 304.2 PP is the ANE-on arm; the separate GPU-only baseline is 242.2 PP and 23.8 TG, and the source calls the decode change within variance. Depth 65536 is not a verified configured maximum context.