Devstral 23.6B , Ollama 2597-token prompt / 4096 configured context
on NVIDIA RTX 5080 · Ollama · 4,096 ctx
Oct 12, 2026
Summary
User reports Devstral 24B at 34.9 t/s generation and 5334.83 t/s prompt processing on an NVIDIA RTX 5080.
Setup is Ollama with Q4_K_M at a configured 4096-token context; the recorded prompt was 2597 tokens, not a full 4096-token prefill.
The figure is a median of 3 fresh-prefill runs at temperature 0 and seed 42, with a unique nonce per repetition to avoid prompt-cache hits; the source captured only 4096-context cells and warns against extrapolating to 8K or 16K.