llamaperf

Qwen3.8 27B

on M5 Ultra 96GB

Apple hardware
Nov 11, 2026
Throughput
95.0 t/s gen · 1180.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
oQ4e
System RAM
96 GB

Summary

User reports Qwen3.8-27B at about 1180 t/s prompt processing and 95 t/s decode on an M5 Ultra Studio with 96GB of RAM. The model runs with MTP speculation and an oQ4e quant; SSD cache is disabled and the hot cache is set to 4GB with a 10% hit rate. System RAM sits just over 50% during heavy work, and the user asks whether a larger hot cache would be reasonable.