llamaperf

Qwen3.8 27B

on M3 Max 128GB · MTPLX · 1,024 ctx

Tone: mixedApple hardware
Oct 7, 2026
Throughput
41.4 t/s gen · 198.2 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
4-bit dynamic/GS32 (MLX)
System RAM
128 GB

Use cases

codinglong-context

Summary

User benchmarks Qwen3.8-27B at 41.4 tok/s decode and 198.2 tok/s prefill at 1K context on an Apple M3 Max 128GB (40-core GPU). Setup is MTPLX v2.8.2 with a 4-bit dynamic/GS32 MLX model, MTP depth 3, single request, 128-token generations, 3-minute GPU rests between rounds. MTPLX decode falls to 19.8 tok/s and prefill to 115.5 tok/s at 16K, where oMLX oQ4e reaches 27.0 tok/s decode and 231.7 tok/s prefill; the 8-bit MTPLX Quality build is nearly identical in speed (18.8 tok/s, 121.3 tok/s) with KL 0.00105 versus Speed's 0.0220. ANE prefill on oMLX gains 57.0% at 32K (126.1 to 198.1 tok/s).