gpt-oss 21B (3.6B active)

on NVIDIA RTX 5080 · Ollama · 16,384 ctx

Oct 12, 2026

Throughput

178.3 t/s gen · 10328.6 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

MXFP4 (GGUF)

VRAM reported

16 GB

Summary

User reports gpt-oss 21B-A3.6B at 178.3 t/s on an NVIDIA RTX 5080. Setup is Ollama with MXFP4 at 16384 configured context, on one 16GB card; the recorded prompt was 1360 tokens, not a full 16384-token prefill. The figure is a median of 3 fresh-prefill runs at temperature 0 and seed 42, with a unique nonce per repetition to avoid prompt-cache hits; it is the Aug 26 re-capture, not the faster Aug 12 capture.