llamaperf

Qwen3.8 27B

on M5 Max 48GB · oMLX · 4,096 ctx

Apple hardware
Oct 25, 2026
Throughput
93.8 t/s gen · 1207.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
4-bit
System RAM
48 GB

Use cases

coding

Summary

User benchmarks oMLX 0.7.2 serving Inco AI's Splash package for Qwen3.8-27B (4-bit target plus DFlash 2 draft) on a MacBook Pro M5 Max with 48 GB, using a Python code context and 128 generated tokens per test. At a 4096-token prompt the single request reaches 93.8 t/s decode and 1207.0 t/s prefill; longer prompts give 189.4 t/s decode at 16382 tokens and 110.2 t/s at 65536 tokens. Batched continuous batching reaches 413.7 t/s aggregate at 2x and 407.0 t/s at 4x, with peak memory between 19.1 GB and 22.2 GB.