Kimi K2.5 1000B (32B active)
8× NVIDIA Tesla P40 24GB · llama.cpp · 262,144 ctx
- generation:
- 5.8 tokens/s
- prompt processing (prefill):
- 10.7 tokens/s
- quant:
- Q2_K_XL (GGUF)
- kv:
- f16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Kimi K2.5 at 5.85 t/s generation and 10.67 t/s prompt processing on eight Tesla P40 24GB cards. Setup is llama.cpp with a Q2_K_XL GGUF (349.01 GiB, 2.92 BPW) at 262144 context, f16 KV cache, flash attention on, experts offloaded to CPU. User says smaller MoE models reach over 20 t/s and Kimi K2.5 gives 5+ t/s; a second request measured 5.47 t/s generation and 7.48 t/s prompt processing.