llamaperf

Qwen3.8 27B

on M3 Ultra 512GB · oMLX · 262,144 ctx

Tone: positiveApple hardware
Sep 28, 2026
Throughput
65-75 t/s gen · 420-442 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
oQ4e (MLX)
System RAM
512 GB

Use cases

agentictool-uselong-context

Summary

User reports Qwen3.8-27B at about 65-75 tok/s generation at 8K context on one Mac Studio M3 Ultra 512GB (one of four run as independent replicas). Setup is omlx 0.6.1 with an oQ4e MLX quant, MTP speculative decoding with 3 draft tokens, and a 262144-token context cap. Prefill measured at 420-442 tok/s; batch-8 throughput reached 274 tok/s. MTP draft depth 3 was the peak (65-75 tok/s vs 50.0 at depth 1 and 65.6 at depth 4). oQ8e was tied on quality but slower at 43 tok/s. A Mac mini M4 Pro 64GB reached 20.8 tok/s and OOM'd around 168k context. An RTX 5090 reference run under llama.cpp with NVFP4+MTP hit ~145 tok/s.