Local model performance on your hardware

generation:

55.7 tokens/s

prompt processing (prefill):

1135.0 tokens/s

quant:

MXFP4 (MXFP4)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks gpt-oss 120B at 55.7 t/s generation and 1135 t/s prompt processing on a Minisforum MS-S1 MAX P495 (Ryzen AI Max+ PRO 495, Radeon 8065S, 192GB). Setup is llama.cpp build 23b0202a1 with Vulkan backend, MXFP4 quant, pp512 and tg128, all layers on GPU. At 32k context the model still reaches 42.3 t/s generation and 385 t/s prompt processing. Other models tested: Qwen3 Coder 30B A3B Q8_0 at 62.8 t/s, Qwen3.8 27B Q8_0 at 8.1 t/s, MiniMax M2.7 230B A10B UD IQ4_XS at 30.2 t/s, and DeepSeek V4 Flash 0731 284B A13B UD IQ3_S at 17.8 t/s. MiniMax crashed at 32k depth with default ubatch and required ubatch 256, which lowered prompt processing.

generation:

17.8 tokens/s

prompt processing (prefill):

238.0 tokens/s

quant:

UD-IQ3_S

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek V4 Flash 284B-A13B at 17.8 t/s generation and 238 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with UD-IQ3_S, Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 238 t/s and 17.8 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output conversation; the same source reports roughly 32k-depth prompt processing at 106 t/s and generation at 13.2 t/s.

generation:

30.2 tokens/s

prompt processing (prefill):

309.0 tokens/s

quant:

UD-IQ4_XS

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports MiniMax M2.7 230B-A10B at 30.2 t/s generation and 309 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with UD-IQ4_XS, Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 309 t/s and 30.2 t/s come from separate llama-bench pp512 and tg128 tests at empty prior context, not a paired 512-input/128-output run; the source also reports about 51 t/s prompt and 14.7 t/s generation at roughly 32k depth.

generation:

8.1 tokens/s

prompt processing (prefill):

466.0 tokens/s

quant:

Q8_0

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 8.1 t/s generation and 466 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with Q8_0 and all layers on the GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 466 t/s and 8.1 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output run; the same source separately reports about 100 t/s prompt and 7.5 t/s generation at roughly 32k depth.

generation:

62.8 tokens/s

prompt processing (prefill):

1692.0 tokens/s

quant:

Q8_0

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-Coder 30B-A3B at 62.8 t/s generation and 1692 t/s prompt processing on an AMD Gorgon Halo 192GB machine. Setup is llama.cpp with Q8_0 and the Vulkan backend, all layers on GPU, on a Minisforum MS-S1 MAX-P495 with Ryzen AI Max+ PRO 495 and 192 GB unified memory. The 1692 t/s and 62.8 t/s come from separate llama-bench pp512 and tg128 tests, not a paired 512-input/128-output run; the same source reports about 205 t/s prompt and 33.1 t/s generation at roughly 32k depth.

generation:

42.3 tokens/s

prompt processing (prefill):

385.0 tokens/s

quant:

MXFP4

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss 120B MXFP4 at 42.3 t/s generation and 385 t/s prompt processing on a Minisforum MS-S1 MAX-P495 with AMD Ryzen AI Max+ PRO 495 and 192 GB unified memory. Setup is llama.cpp build 23b0202a1 with the Vulkan backend and all layers on GPU, at a reported 32k test depth. Figures come from separate llama-bench pp512 and tg128 tests with two repetitions each, not a paired conversation; the source reports 59 GiB model size.