Local model performance on your hardware

prompt processing (prefill):

166.5 tokens/s

quant:

UD-IQ3_S (GGUF)

kv:

F16

flash attention:

on

User reports Mimo 2.5 at 166.5 t/s prompt processing on an AMD Ryzen AI MAX+ 395 with Radeon 8060S graphics. Setup is llama.cpp b9843 with UD-IQ3_S and f16 K/V cache, flash attention on, one participating Radeon. The 166.5 t/s figure is the stock batch2048/ubatch512 arm at depth 512, measured over three repeats of a 2080-token prompt loop; generation throughput and TTFT were not measured.

generation:

37.9 tokens/s

prompt processing (prefill):

301.6 tokens/s

quant:

UD-Q3_K_M (GGUF)

kv:

q8_0

User reports MiMo-V2.5 at 37.87 t/s generation and 301.61 t/s prompt processing on an M2 Ultra 192GB Mac Studio. Setup is llama.cpp llama-server with UD-Q3_K_M GGUF, q8_0 KV cache, 65,536 context, Flash Attention on, gpu_layers 999, batch 2048, and 8,192 MiB cache_ram. Figures are token/time-weighted across four sequential real-world agentic tasks (n=1 each), not llama-bench or end-to-end throughput; one task hit its 8,192-token output cap and 258 of 2,290 prompt tokens were cached. Server peak RSS was 131.015 GiB.

User reports Mimo 2.5 stays fast at large context on dual RTX Pro 6000, using 5-to-1 sliding-window attention similar to Gemma 3. User also mentions Step 3.7 Flash with 3-to-1 sliding-window attention at ~40 t/s at 178k context. User notes MiniMax M3 and DeepSeek V4 have kernel issues on Blackwell consumer GPUs.