171 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX 5090 speeds and models →
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks a new NVFP4 quant of Qwen3.8 27B on an RTX 5090. The quant includes a quantized MTP draft head. User reports it is 50% faster than a Q4 quant and 4-7% faster than other NVFP4 quants.
NVIDIA RTX 5090 · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Kimi K2.5 IQ3_M at 471.40 t/s prompt processing on an RTX 5090, with 80 GB offloaded and the card at 500W. User also tested GLM 5.1 IQ4_NL at 574.98 t/s prompt processing with 70 GB offloaded. The user compares these results with an RTX 6000 PRO MaxQ that has a shunt mod.
NVIDIA RTX 5090 · vLLM
User benchmarks abliteration tools Apostate, Huihui and Heretic on Qwen 2.5 7B on an RTX 5090 32GB. Setup is vLLM 0.19.0 in bf16, evaluated with lm-evaluation-harness. The run reports MMLU, GSM8K, HellaSwag, ARC Challenge, WinoGrande, TruthfulQA MC2, PiQA, LAMBADA ppl, HarmBench ASR and KL divergence. No tokens/sec is reported.
NVIDIA RTX 5090 · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks an FWHT CUDA implementation for KV cache quantization on Gemma 4 26B.A4B Q4_K_M. Setup is llama.cpp with -ctk q8_0 -ctv q8_0, reporting pp2048 and tg128 values. The implementation gives a 1-2% pp boost and a 7-9% tg boost, with the highest t/s taken from the cuda-fwt column.
User reports inconsistent output from local models Gemma 4 and Qwen 3.6 35B A3B compared to frontier models, for a dictation and SOAP scribe. Setup is a Core Ultra 9 with 128 GB RAM and an RTX 5090, running Proxmox with AnythingLLM and Ollama on llama.cpp. The system prompt is a 25-30k token markdown file.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 228 t/s baseline and 578 t/s with DFlash speculative decoding, a 2.56x speedup, on vLLM 0.19.2rc1. Setup uses the z-lab/gemma-4-26B-A4B-it-DFlash draft model, with 256 input tokens and 1024 output tokens.
NVIDIA RTX 5090 · vLLM · 200,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports generation speed varying 59-111 tok/s with MTP enabled and 3 speculative tokens. Setup uses an fp8_e4m3 KV cache, with prefix caching tested. Mean MTP acceptance length is 2.28. A stability pass at 200k context completed 10/10 runs.
NVIDIA RTX 5090 · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a 6% speed increase from MTP on a grafted model at Q4 on an RTX 5090. User also tests Q8 on an RTX 5090 plus an RTX 3090, reaching 148.20 t/s without MTP and 152.02 t/s with MTP.
NVIDIA RTX 5090 · 50,000 ctx
User reports nvidia/Gemma-4-26B-A4B-NVFP4 running on an RTX 5090 with 80% of the 32 GB allocated, reaching around 50k context. The model is 18.8 GB. Benchmarks compare baseline against NVFP4: GPQA Diamond 80.30% vs 79.90%, AIME 2025 88.95% vs 90.00%, MMLU Pro 85.00% vs 84.80%, LiveCodeBench pass@1 80.50% vs 79.80%, IFBench 77.77% vs 78.1%, and IFEval 96.60% vs 96.40%.
NVIDIA RTX 5090 · vLLM · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B-INT4 at 105-108 t/s generation on an RTX 5090 at 256k context. Setup is vLLM 0.19 with the Lorbus AutoRound quant, an fp8_e4m3 KV cache, flashinfer attention, and MTP speculative decoding with 3 tokens.
NVIDIA RTX 5090 · vLLM · 218,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B at ~80 t/s on an RTX 5090 at 218k context. Setup is vLLM 0.19.1rc1 with NVFP4 quantization on a single card.