llamaperf

Devstral 23.6B , Ollama 2597-token prompt / 4096 configured context

on NVIDIA RTX 5080 · Ollama · 4,096 ctx

NVIDIA hardware
Oct 12, 2026
Throughput
34.9 t/s gen · 5334.8 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
VRAM reported
16 GB

Summary

User reports Devstral 24B at 34.9 t/s generation and 5334.83 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at a configured 4096-token context; the recorded prompt was 2597 tokens, not a full 4096-token prefill. The figure is a median of 3 fresh-prefill runs at temperature 0 and seed 42, with a unique nonce per repetition to avoid prompt-cache hits; the source captured only 4096-context cells and warns against extrapolating to 8K or 16K.