Local model performance on your hardware

generation:

22.9 tokens/s

prompt processing (prefill):

2567.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

User benchmarks MiniMax-M3.1-Flash 428B at 22.9 t/s decode and 2,567 tok/s prefill at 64K context on 8x NVIDIA A40 with TensorSharp. Setup is TensorSharp with Unsloth UD-Q4_K_XL GGUF (232.3 GiB of model files), F16 KV cache, layer splitting, and speculative decoding disabled. Against Unsloth's llama.cpp support branch on the same files, 64K prefill was 262 tok/s and decode 15.4 t/s; short-chat decode was nearly tied at 26.3 vs 26.0 t/s. Tensor parallelism reached about 41 t/s short-chat decode, and DSpark with confidence gating raised mean generation from 22.9 to 32.3 t/s.

generation:

22.9 tokens/s

prompt processing (prefill):

2567.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

User benchmarks MiniMax-M3.1-Flash at 22.9 tok/s decode at 64K context on 8x NVIDIA A40 GPUs. Setup is TensorSharp with the Unsloth UD-Q4_K_XL GGUF, F16 KV cache, layer splitting, and no speculative decoding. Prefill was 1,699 tok/s at 8K, 2,449 tok/s at 32K and 2,567 tok/s at 64K; time to first token on a 64K prompt was 40.0 s. Results are early, from one host and one quantization, and are compared against llama.cpp (llama-server with flash attention and --no-kv-unified).