llamaperf

Qwen3.8 27B

on M2 Ultra 128GB · oMLX

Apple hardware
Oct 8, 2026
Throughput
23.0 t/s gen · 304.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
oQ4e (MLX)
System RAM
128 GB

Use cases

summarizationagenticlong-context

Summary

User reports Qwen3.8 27B at 23 t/s generation and 304.2 t/s prompt processing on an M2 Ultra 128GB Mac Studio. Setup is oMLX 0.6.3 with oQ4e 4-bit quantization at workload depth 65536, with 16.7GiB of weights and peak memory rising from 25.3 to 36.2GB. The 304.2 PP is the ANE-on arm; the separate GPU-only baseline is 242.2 PP and 23.8 TG, and the source calls the decode change within variance. Depth 65536 is not a verified configured maximum context.