llamaperf

M5 Ultra 96GB

APPLE · 96GB unified memory · 12 reports

Is this your card?

On the M5 Ultra 96GB, Qwen3.8 27B at 8-bit, the model most run on it, generates a median of 15.0 tokens per second from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the M5 Ultra 96GB, each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

Bars are medians on one scale, 0 to 15.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: Inco Splash 4 (reports) · mlx-serve 3 (reports) · Ollama 1 (reports) · oMLX 1 (reports)

Run models on your M5 Ultra 96GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the M5 Ultra 96GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 96 GB of unified memory. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: positiveApple hardware
generation:
186.9 tokens/s
quant:
IQ3_S (GGUF)
agentictool-use

User reports Qwen3.8 Flash-Next at 186.9 tok/s on an M5 Ultra Mac Studio with 96GB of RAM under Splash 1.6.1. Setup is Splash with IQ3_S weights, run as a local assistant for editing video in DaVinci Resolve Studio via MCP. The user says the release is a significant improvement in both performance and concurrency, with the peak moving from a width of 4 to a width of 12.

Nov 11, 2026
Tone: mixedApple hardware
generation:
30-62 tokens/s
prompt processing (prefill):
915-1800 tokens/s
quant:
STQ-1.75bpw
long-contexttool-use

User reports DeepSeek V4.1 Flash running at 1M-token context on a 96 GB Apple M5 Ultra, with decode of ~30–62 tok/s and prefill of ~915–1,800 tok/s. Setup is Sushi 1.5.1 on MLX 0.32.3 with a 1.75bpw STQ pack (~310 GB on disk), streaming routed experts from SSD with an 88 GB SSD budget; the 189 GB Engram tables are read from SSD and never loaded into GPU memory. First token takes 1.2–70 s from 1k to 128k context, and ~17 min cold at 1M where decode falls to ~23 tok/s. User calls it clearly smarter than the Qwen 3.8 Flash Next 2.75 quant they run daily, but too slow for daily chat or coding loops; the Qwen comparison is from a different day and protocol.

Nov 11, 2026
Apple hardware
generation:
95.0 tokens/s
prompt processing (prefill):
1180.0 tokens/s
quant:
oQ4e

User reports Qwen3.8-27B at about 1180 t/s prompt processing and 95 t/s decode on an M5 Ultra Studio with 96GB of RAM. The model runs with MTP speculation and an oQ4e quant; SSD cache is disabled and the hot cache is set to 4GB with a 10% hit rate. System RAM sits just over 50% during heavy work, and the user asks whether a larger hot cache would be reasonable.

Nov 11, 2026
Tone: mixedApple hardware
generation:
119.0 tokens/s
quant:
6bit
codingagentic

User compares a 6-bit fine-tune of Qwen3.8 27B called sabi-noema against Qwen3.8 Flash-Next on an M5 Ultra Mac Studio 96GB, with MTP enabled for both. The 27B fine-tune reached 119 tok/s against 90 tok/s for Flash-Next, and the user notes better performance on Splash for the 27B. Flash-Next used 2.5x the memory with N-gram offload, 73GB peak versus 29GB peak. The user found the 27B better at coding tests and prose, while Flash-Next did better on broad knowledge.

Nov 7, 2026

Qwen3.8 27B

M5 Ultra 96GB · Ollama · 6,600 ctx

Tone: mixedApple hardware
generation:
52.9 tokens/s
prompt processing (prefill):
1168.0 tokens/s
quant:
MXFP8 (MLX)
kv:
q8_0
codingtool-useagentic

User reports Qwen3.8-27B at 52.9 t/s decode on a 6.6K prompt and 1,168 t/s prompt eval on a Mac Studio M5 Ultra 96GB (64-core GPU), compared against oMLX at 23.3 t/s cold and 29.7 t/s warm on the same model. Setup is Ollama 0.40.13 with the MLX runner, MXFP8 weights, q8_0 KV cache and a 131072 context; the wired memory limit was raised to 86GB. Other Ollama short-prompt speeds: gpt-oss:120b 95 to 102 t/s, qwen3-coder-next 107 t/s, qwen3.6:35b-a3b 246 t/s, llama4:scout 49 t/s. On long prompts gpt-oss prefill was 2,588 t/s at ~15K and 1,996 t/s at ~60K, Coder-Next 3,382 and 2,687 t/s, Qwen3.8 1,376 and 1,243 t/s. The user expected oMLX to win and it did not, and notes Qwen3.8's thinking volume (10,339 tokens, 161 s) as a real cost.

Nov 1, 2026
Tone: positiveApple hardware
generation:
90.0 tokens/s
prompt processing (prefill):
1260.0 tokens/s
quant:
oQ3.5e (MLX)
codingagenticlong-context

User reports Qwen3.8 Flash-Next at 90 tok/s generation and 1,260 tok/s prompt reading on an M5 Ultra Mac Studio with 96 GB, running as a coding agent at 262k context. Setup is oMLX 0.7.2 with the oQ3.5e (~3.5-bit) MLX quant, 60 GB model footprint, 74 GB average and 83 GB peak memory, 2.1 s median time to first token. With Splash as the engine, Flash-Next reached 140 tok/s at Q2_0, 129 tok/s at IQ3_XXS, 132 tok/s at IQ3_S and 79 tok/s at UD-IQ4_XS; Qwen3.8 27B ran at 56 tok/s (oQ8e) and 92 tok/s (UD-Q4_K_M). Two concurrent agents gave 1.52x work per hour at 93 tok/s each, three gave 1.56x at 72 tok/s each. The user notes oMLX crashed out of GPU memory with MTP enabled and multiple agents, and that Flash-Next keeps a ~29 GB lookup table on the SSD.

Oct 29, 2026
Apple hardware
generation:
145.6 tokens/s
prompt processing (prefill):
3880.0 tokens/s
quant:
UD-IQ4_XS (GGUF)
tool-use

User benchmarks Qwen3.8 Flash-Next at 145.6 t/s decode and 3,880 t/s prefill at 1k context on an M5 Ultra 96GB, using Inco Splash 1.5.0 with the UD-IQ4_XS GGUF and an auto-paired DFlash2 drafter. Setup streams weights from disk at about 78 GB resident with 262k context; the 88 GB GGUF sits in the HF cache. Decode falls to 115.4 t/s at 4k and 112.2 t/s at 16k, then reads 163.0 t/s at 32k, 139.8 t/s at 64k and 135.6 t/s at 128k. Figures are medians of 3 reps with unique-nonce prompts and one request in flight; the machine was also serving an agent chat throughout. The post compares Splash against Sushi 2.75bpw, whose numbers are not given here.

Oct 29, 2026
Tone: positiveApple hardware
generation:
144.6 tokens/s
prompt processing (prefill):
3444.0 tokens/s
quant:
IQ3_S (GGUF)
codingagentictool-uselong-context

User reports Qwen3.8 Flash-Next (Swift 1.5) at 144.6 tok/s decode and 3,444 tok/s prefill on a Mac Studio M5 Ultra 96GB, at a median context of 147.5K tokens. Setup is Splash with IQ3_S GGUF weights and a 262,144-token context window, driven by the DeepSeek Harness agent framework for agentic coding. With two concurrent requests combined decode is 200.4 tok/s (100.2 per request) and prefill 3,006 tok/s; with three it is 218.0 tok/s combined (72.7 per request) and 3,253 tok/s prefill, a result the user calls preliminary. The user is happy with the numbers and finds the model less lazy than Sol at troubleshooting.

Oct 26, 2026
Apple hardware
prompt processing (prefill):
1354.0 tokens/s
quant:
iQ-MLX 3.3 bpw (MLX)
codingagenticlong-context

User reports Qwen3.8-Flash-Next on an M5 Ultra 96GB (64-core GPU) prefilling a cold 3.7k-token prompt at 1,354 tok/s. Setup is mlx-serve 26.10.1 with iQ-MLX 3.3 bpw, 131,072 context, native MTP adaptive decoding, PLD and a hot prefix cache, one request at a time. Decode is given per workload only: 227.5 tok/s on copy, 172.8 on new code and 106.4 on novel prose, with speculation on. Cold prefill rose to 4,083 tok/s at 25.2k. An agent replay ran at 125.8 tok/s text-round decode.

Oct 8, 2026

Qwen3.8 27B Swift-1.5

M5 Ultra 96GB · mlx-serve · 25,000 ctx

Tone: positiveApple hardware
generation:
113.7 tokens/s
prompt processing (prefill):
3191.0 tokens/s
quant:
4.7bpw (MLX)

User reports Qwen3.8 Swift-1.5 at 113.7 tok/s decode and 3191 tok/s prefill on an M5 Ultra 96GB Mac Studio. Setup is mlx-serve 26.10.1 with a 4.7bpw MLX quantization, 107GB download, text-only, tested to 179,200 tokens of context. Decode falls to 81.1 tok/s after a 95k prompt, where prefill is 2928 tok/s. User compares against a llama.cpp IQ3_XXS build of the same model, which reaches 62.7 tok/s decode after 4k and 1427 tok/s prefill at 25k. Top-1 agreement with Swift BF16 is 91.0% across 680 held-out positions, against 84.1% for the llama.cpp build.

Oct 6, 2026
Tone: positiveApple hardware
generation:
42.7 tokens/s
prompt processing (prefill):
828.2 tokens/s
quant:
4-bit-8bit mix (MLX)
kv:
8-bit
codingagenticlong-context

User reports Qwen3.8 Flash-Next on a base M5 Ultra 96GB (64-core GPU) at a median 828.2 t/s prefill and 42.7 t/s decode per stream, with a single-stream max of 3,328 t/s prefill and 152 t/s decode. Setup is a custom mlx-serve build with continuous batching at 4-way concurrency, a 4-bit/8-bit mixed MLX quant with MTP, 4 x 128k context (512k total) and 8-bit KV cache using 90GB of unified memory. Aggregate throughput was ~3,200 t/s prefill and ~170.8 t/s decode at 4-way concurrency. The run covered 112M tokens (109M prompt, 3M generated) with an 89% cache hit rate across ~1,700 sub-agent calls.

Sep 23, 2026
Apple hardware
generation:
15.0 tokens/s
quant:
Q8
agentic

User compares an M5 Ultra 96GB against an M5 Max 128GB for running Qwen3.8-27B at Q8, and reports about 15 t/s on the Ultra. The user also discusses Qwen3.8-Flash-Next, a multimodal MoE with 176B total parameters and about 6B active, and estimates it will not fit in 96GB. MLX and llama.cpp are mentioned as possible engines, but the user does not confirm using either.

Sep 7, 2026

Get a weekly email of new M5 Ultra 96GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3.8 125B (6B active) Flash-Next
M5 Ultra 96GB
IQ3_S
Inco Splash
Not reported186.9 tokens/s
Qwen3.8 27B
M5 Ultra 96GB
oQ4e
Engine not reported
Not reported95.0 tokens/s
Qwen3.8 27B sabi-noema
M5 Ultra 96GB
6bit
Inco Splash
Not reported119.0 tokens/s
Qwen3.8 27B
M5 Ultra 96GB
MXFP8
Ollama
6,60052.9 tokens/s
Qwen3.8 125B (6B active) Flash-Next
M5 Ultra 96GB
oQ3.5e
oMLX
262,14490.0 tokens/s
Qwen3.8 125B (6B active) Flash-Next
M5 Ultra 96GB
UD-IQ4_XS
Inco Splash
1,024145.6 tokens/s

Frequently asked

How fast is the M5 Ultra 96GB for local LLMs?

On the M5 Ultra 96GB, Qwen3.8 27B at 8-bit, the model most run on it, generates a median of 15.0 tokens per second from one plain run (community reports, as of 22 Nov 2026).