NVIDIA RTX 5080

prompt processing (prefill):

5500.0 tokens/s

quant:

IQ3_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next at up to 275 tok/s peak generation and 5500 tok/s prompt processing on a single RTX 5080 16GB with 64 GB DDR5-6400 and a Core Ultra 9 285K. Setup is the Strata fork Basalt with IQ3_S weights, run during a live opencode coding session. Code and tool calls peak at 275 tok/s while reasoning traces reach up to 190 tok/s; the user says the IQ3_S quant is very usable.

generation:

40.0 tokens/s

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at about 40 tokens/second on an RTX 5080 plus RTX 5070, with MTP disabled. Setup is Ollama with Q8_0 weights and 128k of context; the vision encoder and MTP were turned off to fit the split across the two cards. With MTP enabled at 64k context the user sees about 54 tokens/second. They ask whether Ollama's per-GPU buffer can be lowered, having had no luck with LLAMA_ARG_FIT_TARGET, or whether to switch to llama.cpp.

generation:

130.6 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User compares an RTX 5080 desktop against a rented DGX Spark on Qwen3-8B Q4_K_M, where the PC decodes at 130.55 t/s versus 42.43 t/s on Spark at 512 input tokens, roughly three times faster. Both ran llama.cpp release b11889 with Q8 KV cache, Flash Attention and a 40,960-token context; the PC kept the whole model on its 16 GB card. Decode falls to 79.09 t/s on the PC and 25.64 t/s on Spark at 32,768 input tokens. With Qwen3-32B the PC needed CPU offload (40 GPU layers) and managed 4.92 t/s against Spark's 10.50 t/s. Aggregate batched throughput at 8 concurrent requests was 471.75 t/s on the PC and 209.02 t/s on Spark.

generation:

154.9 tokens/s

prompt processing (prefill):

5475.0 tokens/s

quant:

IQ3_S (GGUF)

kv:

int8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next IQ3_S on an RTX 5080 16 GB with a Core Ultra 9 285K and 62 GB DDR5-6800, averaging 154.9 t/s generation and 5,475 t/s prompt processing in a context sweep of fresh 15K-207K prompts. Setup is Basalt with IQ3_S weights, 262K context and int8 KV cache, using 15.7 GB VRAM and 54.1 GB system RAM. An agent coding session over 103 requests reached 153.0 t/s average generation with a 262 t/s peak, and 5,011 t/s average prefill; 99.5% of prompt tokens came from cache with a 0.2 s median time to first token.

generation:

150.0 tokens/s

prompt processing (prefill):

4396.0 tokens/s

quant:

IQ3_XXS (GGUF)

kv:

int8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 150 tok/s decode and about 4,400 tok/s prompt on a single RTX 5080 16 GB. Setup is Strata v0.2.1 with IQ3_XXS GGUF, int8 KV cache and 64K context, MTP speculation on, 15 CPU expert workers and an expert cache of 5,310 experts (8.61 GiB VRAM). Decode median 150.0 tok/s over 9 runs (126.7-170.2); fresh prompt runs gave 3,755 tok/s at 4K, 4,396 at 16K and 4,412 at 32K. The card was tuned with LACT (memory +6000, core capped 2850 MHz, 360 W), so stock cards may be slower.

generation:

89-112 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User compares Qwen3.8 Flash-Next on Strata against a Qwen3.8 27B build for the RTX 5080 on NInfer, reporting 89-112 t/s decode on the 27B. Setup is NInfer on an RTX 5080 with 32GB DDR5, vision flags removed and thinking budget raised to 4096; the user plugged the display into the iGPU and reports 94.6% VRAM usage while using the desktop. The user says the 27B solved in one prompt a coding task that the Flash-Next coder and an IQ2_XS build needed multiple prompts for, and calls it insanely fast at prefill and decode.

generation:

34.9 tokens/s

prompt processing (prefill):

5334.8 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Devstral 24B at 34.9 t/s generation and 5334.83 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at a configured 4096-token context; the recorded prompt was 2597 tokens, not a full 4096-token prefill. The figure is a median of 3 fresh-prefill runs at temperature 0 and seed 42, with a unique nonce per repetition to avoid prompt-cache hits; the source captured only 4096-context cells and warns against extrapolating to 8K or 16K.

generation:

51.7 tokens/s

prompt processing (prefill):

1928.1 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3-Coder 30B-A3B at 51.7 t/s generation and 1928.1 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at a configured 16384-token context, with partial CPU/RAM offload since the model exceeds VRAM. The recorded prompt was 1347 tokens, not a full 16384-token prefill, and the figures are medians of 3 fresh-prefill runs; 74% GPU residency was measured separately at 8K context, not this cell.

generation:

93.4 tokens/s

prompt processing (prefill):

4842.2 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5 Coder at 93.4 t/s generation and 4842.2 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at 16384 tokens of configured context, on one 16GB card. The recorded prompt was 1368 tokens, not a 16384-token prefill, and the figures are medians of 3 fresh-prefill runs at temperature 0 with a 256-token target.

generation:

171.4 tokens/s

prompt processing (prefill):

9825.5 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5 Coder-7B at 171.4 t/s generation on one NVIDIA RTX 5080 16GB. Setup is Ollama with Q4_K_M at 16384 tokens of configured context, on a Ryzen 7 7800X3D under Windows 11. The figure is the median of 3 fresh-prefill runs with a 1368-token prompt, not a 16384-token prefill; prompt processing was 9825.5 t/s.

generation:

176.6 tokens/s

prompt processing (prefill):

9984.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5 7B at 176.6 t/s generation and 9984.4 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at a configured 16384-token context, on one 16GB card. The recorded prompt was 1674 tokens, not a full 16384-token prefill, and the figures are medians of 3 fresh-prefill runs.

generation:

298.6 tokens/s

prompt processing (prefill):

19397.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Llama 3.2 3B at 298.6 t/s generation and 19397.4 t/s prompt processing on an NVIDIA RTX 5080. Setup is Ollama with Q4_K_M at a configured 16384-token context, with an actual recorded prompt of 1657 tokens rather than a full 16384-token prefill. The figures are medians of 3 fresh-prefill runs, derived from Ollama eval_count/eval_duration rather than TTFT division, with temperature 0 and a unique nonce per repetition to avoid prompt-cache hits.

generation:

94.6 tokens/s

prompt processing (prefill):

4864.7 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen2.5 14B at 94.6 t/s generation and 4864.7 t/s prompt processing on one NVIDIA RTX 5080 16GB. Setup is Ollama with Q4_K_M at a configured 16384-token context; the recorded prompt was 1674 tokens, not a full 16384-token prefill. The figures are medians of 3 fresh-prefill runs, with 256 target output tokens and a unique nonce per repetition to avoid prompt-cache hits.

generation:

178.3 tokens/s

prompt processing (prefill):

10328.6 tokens/s

quant:

MXFP4 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports gpt-oss 21B-A3.6B at 178.3 t/s on an NVIDIA RTX 5080. Setup is Ollama with MXFP4 at 16384 configured context, on one 16GB card; the recorded prompt was 1360 tokens, not a full 16384-token prefill. The figure is a median of 3 fresh-prefill runs at temperature 0 and seed 42, with a unique nonce per repetition to avoid prompt-cache hits; it is the Aug 26 re-capture, not the faster Aug 12 capture.

generation:

30.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at about 30 tok/s on an RTX 5080 with two RTX 5060 Ti 16GB cards in a multi-GPU setup. Setup is LM Studio with the default Qwen parameters; the two secondary cards run at PCIe 4.0 x4. The user asks whether swapping in an RX 7900 XT or switching to Vulkan would improve speed, and whether a Thunderbolt 3 eGPU would only affect model loading time.

generation:

80-100 tokens/s

quant:

IQ2_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B IQ2_M at 80-100 tok/s on an RTX 5080 with 64GB system RAM. Setup is LM Studio with IQ2_M quant; the model is partially offloaded to system RAM because the 16GB card has about 1.5GB used at idle. The user finds the model too slow for web-search automation and says it loops on web search until hitting the context limit without retrieving the price.

generation:

96.5 tokens/s

kv:

q4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen 3.8 27B at a median 96.45 t/s decode on a 16GB RTX 5080, with a range of roughly 90-110 t/s during coding tasks. Setup is NInfer v1.5 with q4 KV cache, MTP speculative decoding with 3 draft tokens, 110,592 context allocated, and 11.86 GiB of weights leaving about 498 MiB of slack. Across 32 completed requests the lowest decode was 84.6 t/s and the highest 131.7 t/s, with median time to first token 1.4 seconds. About 68% of generated tokens were reasoning tokens, and one request that lost its prompt cache took almost 50 seconds to process a 79k prompt before generating at about 94 t/s.

generation:

110.6 tokens/s

prompt processing (prefill):

2773.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

INT8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 110.6 t/s generation and 2773 t/s prefill on two RTX 5080s with a Ryzen 9 9900X, using a custom Strata fork. Setup is Strata with UD-Q4_K_XL, 64K context, INT8 K/V, 8K prompt chunk, MTP on and expert adaptation on. Upstream v0.1.38 reached 78.1 t/s generation and 2016 t/s prefill with layer split, and 70.3 t/s generation and 1522 t/s prefill with the host peer tier. With static caches custom was about 26% ahead of upstream layer split (55 vs 44 t/s). Letting the second GPU compute experts during prompt processing doubled custom's 32K prefill (1366 to 2731 t/s), and a 16K chunk brought it to 3724 t/s. Putting the expert tier on the x8 PCIe card improved 4K prefill by 58% and 32K by 34%. The saved answer came from the custom fork, so it may favor its drafts.

prompt processing (prefill):

1940.0 tokens/s

quant:

IQ2_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next (Swift 1.5 IQ2_XS) at 143 tok/s code generation on an RTX 5080 + RTX 4060 Ti with 32 GB system RAM. Setup is a dual-GPU fork of the Strata engine at 256K context, with experts that don't fit in VRAM read from the SSD and a fine-tuned draft layer for speculative decoding. The same machine with the 5080 alone reached 29 tok/s code and 333 tok/s on a 32K prompt; the fork with both cards reached 1,940 tok/s on the 32K prompt. In real use with the Pi coding agent it peaks above 200 tok/s (209 so far) and a 150K-token context reads at about 1,850 tok/s. Perplexity was 7.24 vs upstream's 7.28.

generation:

80.0 tokens/s

prompt processing (prefill):

1200.0 tokens/s

quant:

Q3_K_XL (GGUF)

kv:

q8_0

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 80+ t/s generation and 1200+ t/s prompt processing on an RTX 5080 with 16GB VRAM, using CUDA unified memory to fit the model. Setup is llama.cpp with the Unsloth UD Q3_K_XL GGUF, 160000 context, q8_0 KV cache, flash attention, and MTP speculative decoding with ngram-mod draft. User asks whether Q3 quants are good enough for harness coding with opencode, pi, and qwen-cli, and notes this is the only quant that fits the whole model in VRAM.