gpt-oss 21B (3.6B active)
on NVIDIA RTX 3090 · 128,000 ctx
Use cases
tool-use
Summary
User reports gpt-oss-20b at close to 200 tps aggregate across parallel requests with the full 128k context on a single RTX 3090, under a custom harness (burrito) that fixes the model's tool calling; 320,192 evals over 8 seeds and 3.49B tokens took 1,062 GPU hours at batch size 1.
Setup is factory-precision gpt-oss-20b weights on one RTX 3090 serving parallel requests; no single-stream figure and no decoding method is given.
Effort levels affect accuracy: Low 38.3%, Medium 97.1%, High 100.0%. The user mentions llama.cpp and vLLM had issues with the model.