171 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX 5090 speeds and models →
NVIDIA RTX 5090 · vLLM
User benchmarks Gemma4-12B on an RTX 5090, comparing vLLM stock, vLLM+Emmy, and llama.cpp. The runs report throughput and latency across various token lengths and concurrency levels. The comparison also covers speculative decoding (MTP) tests and kernel optimization details.
2× NVIDIA RTX 5090 · llama.cpp · 384,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Q8_K_XL against IQ3_XXS on DeepSeek-V4-Flash-0731-UD. Q8_K_XL offloads about 40 GB to system RAM. IQ3_XXS is 2.2x faster at decode and 2.4x faster at prefill, with equal correctness. Q8_K_XL has more empty-answer failures, 8/125 against 5/125. A reasoning budget of 4096 eliminates empty answers, and disabling reasoning degrades trap handling. Q8_K_XL is slower but more reliable in some tasks.
NVIDIA RTX 5090 · vLLM · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports full 1M context on a single RTX 5090 with CPU/RAM offloading, at about 800 t/s prefill and over 15 t/s decode. Setup uses DSpark speculative decoding with dynamic acceptance rates, with two routed MoE layers kept GPU-resident.
8× NVIDIA RTX 5090 · 250,000 ctx
User reports 2000+ t/s aggregate and 30-40 t/s per session across 50 concurrent sessions on 8x RTX 5090. Setup is the REAP mxfp4 image with 250k KV per session.
NVIDIA RTX 5090 · vLLM · 1,048,576 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DSpark K1/K2 at 13.78 t/s for reasoning and 17-18 t/s for content on a single RTX 5090 at full 1M context. Setup is a vLLM fork, guqiong96/Lvllmds4-x reporting vLLM 2.3.9, with phase-adaptive K1/K2 dual CUDA graphs and routed MoE experts in system RAM.
8× NVIDIA RTX 5090 · llama.cpp · 5,632 ctx
User compares quants of DeepSeek V4 Flash 0731 on 8x RTX 5090, benchmarking with wikitext-2 at 5632 context and reporting PPL and top-1 accuracy. The quant named AD-IQ2_M is 104 GB and reaches 83.6% top-1. The user mentions an H100 for comparison.
NVIDIA RTX 5090 · NInfer
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at about 200 t/s generation on a single RTX 5090 with speculative decoding, on day-0 support in NInfer. The setup uses a shared paged KV cache and ReplaySSM for GDN, with PDL in use. The user notes support for up to 8 concurrent requests.
NVIDIA RTX 5090 · SGLang
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-27B at over 200 t/s decode on an RTX 5090 and an RTX Pro 6000, and 38 t/s on a DGX Spark. Setup is NVFP4 with DSpark on day-0 support. The user also mentions an H200 in the cookbook link.
NVIDIA RTX 5090 · llama.cpp · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares Qwen3.8-27B against Qwen3.6-27B on an RTX 5090. The user reports prefill and decode speeds at concurrency 1, and finds the subjective quality improvement smaller than benchmarks suggest. Medium reasoning used fewer tokens than low on 3 of 5 prompts. The concurrency knee sits at 3 requests for Qwen3.8 versus about 7 for Qwen3.6.
2× NVIDIA RTX 5090 · vLLM
User benchmarks 200 prompts at max concurrency 100, reaching 15,058.74 tok/s total token throughput and 1,673.19 tok/s output token throughput, with a peak output of 3,000 tok/s. Setup is vLLM with Model Runner V2, p2p enabled, and tensor parallel 2. Mean TTFT is 1,741.86 ms and mean TPOT is 45.74 ms. An AMD Radeon PRO W6600 is mentioned but not used for this benchmark.
4× NVIDIA RTX 5090 · 4,096 ctx
User compares GGUFs of Qwen3.8-27B on 4x RTX 5090. Quants mentioned are Q8_0, IQ1_M, Q3_K_M, AD-IQ3_S and Q6_K, with KLD values reported. No tokens per second are given.
NVIDIA RTX 5090 · llama.cpp · 65,536 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports that DSpark GGUF for Qwen3.8 27B shows no performance improvement and is extremely memory hungry. Setup is llama.cpp with DSpark support merged in PR #25173, using a converted and quantized RadixArk/Qwen3.8-27B-DSpark GGUF, with context set to 64k to test the BF16 speculator. Draft acceptance is 0.28922 with a mean length of 3.02.
NVIDIA RTX 5090 · vLLM
User reports KLD against BF16 for custom EXL3 quants of Qwen3.8-27B: 0.007406 for EXL3, 0.013126 for FP8, and 0.094978 for NVFP4. The quants require a custom vLLM fork, Gilded Gnosis.
NVIDIA RTX 5090 · llama.cpp · 137,984 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks context length against KV cache quants on an RTX 5090 with Qwen3.8 27B MTP Q8_0 GGUF. Maximum stable context is 105,216 tokens with a Q8_0 KV cache, 137,984 with Q5_1, and 169,984 with Q4_0. Throughput dropped from 100 t/s to 25 t/s, which the user attributes to a llama.cpp issue.
NVIDIA RTX 5090 · NInfer · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports an aggregate 880 t/s at 6 parallel requests, peaking at 967 t/s, and over 200 t/s single stream with MTP speculative decoding. Prefill runs at about 5,950 t/s. Weights take 16.8 GiB, leaving about 13 GiB for KV cache. Benchmarks are HumanEval+ 152/164 and AIME25+AIME26 55/60, identical to int4 but 1.56x-1.98x faster. The run requires Blackwell FP4 cores and a 6-line patch not yet upstream.
NVIDIA RTX 5090 · vLLM · 65,536 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at 59 t/s single stream on an RTX 5090, with aggregate throughput above 100 t/s for two requests and 279 t/s with four parallel jobs. Setup is vLLM 0.27.1 with the PrismaAqua 5.5-bit quant. The user also mentions the RTX 6000 Pro as target hardware. Quality is close to BF16 on a custom suite. The user prefers medium reasoning over xhigh because of fewer token-cap failures.
NVIDIA RTX 5090 · NInfer
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports median decode speeds by context size on an unspecified setup: 169.7 t/s under 50k, 167.6 t/s at 50-100k, 156.9 t/s at 100-150k, 149.0 t/s at 150-200k, and 143.7 t/s at 200k and above. Context length is set to maximum. Peak decode for a single request is 222.0 t/s, with 211.2 t/s sustained over 5s.
NVIDIA RTX 5090 · llama.cpp · 160,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DFlash2 speculative decoding with Qwen3.8 27B on an RTX 5090, with bursts of ~200 tk/s on code, ~80-90 tk/s on thinking, and an average of ~120 tk/s for code generation. Setup uses a Q5_K_L base model and a Q4_K_M draft model. The user compares this to MTP at ~140 tk/s on code and ~100 tk/s otherwise, notes the setup is memory hungry and required reducing context from 220k to 160k, and reports better results previously with DFlash on Qwen3.6 27B.
NVIDIA RTX 5090 · llama.cpp · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a DFlash2 draft model with a Q4_K_M quant, giving roughly 2x decode speedup with 5 draft tokens and roughly 2.7x with 7 draft tokens. Setup is a 4-bit Unsloth model at 256k context, with draft acceptance identical to Q8_0. Prefill slows by roughly 15% with the draft model.
NVIDIA RTX 5090 · SGLang
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports the Escha 2-bit quant of Qwen3.8-27B, 10.15 GB on disk, averages about 100% of FP8 performance across 8 benchmarks. On LiveCodeBench v6, Escha W2 scores 86.81 against 85.16 for FP8. The previous 2-bit Qwen3.6-35B MoE scored 62.6 against 67.0 for FP8 on the same benchmark.