NVIDIA RTX 4080 Super

generation:

10.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B running at 10 tokens per second on an RTX 4080 Super with an i9-14900K and 64GB of RAM. Setup is LM Studio serving over the local network, with the model downloaded through the app; quant, context length and KV cache are not stated. The user is asking what speed to expect as a baseline and how to run the model more effectively, and also wants image input support.

generation:

33.1 tokens/s

quant:

Q3_K_XL (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B decode throughput collapsing from 33.1 t/s at 68K position to 1.4 t/s at 91K position on an RTX 4080 Super 16GB. Setup is llama.cpp (master 8144f31) with UD-Q3_K_XL GGUF and Q8_0 KV cache, no flash attention, 73728 context, single GPU with zero CPU tensor overrides. Prompt processing stays fast at ~1300 t/s at the same 91K position. The user notes the cliff is positional rather than a memory-spill issue, reproduced across q4_0/q5_0/q8_0 KV quants and with FA on/off, and also under Windows native b10488.

generation:

40-60 tokens/s

prompt processing (prefill):

1000.0 tokens/s

quant:

q3_xss (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports running Qwen3.8 Flash-Next at q3_xss on an RTX 4080 Super with a 9800X3D and 64 GB DDR5 6000, getting 1000 t/s prefill and 40-60 t/s generation. Setup uses the Strata engine. The user questions whether the speed increase is too good to be true and suspects the model may actually be Qwen3.6 35B A3B. The user notes the model does not think very much and feels quite dumb.

generation:

12.9 tokens/s

prompt processing (prefill):

355.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 12.9 tok/s decode and 355 tok/s prefill at 155K context on an RTX 4080 Super 16 GB. Setup is the QwFNfer engine with UD-Q4_K_XL weights and q8_0 KV cache; the dense core and a VRAM expert tier stay on the card while the remaining experts are read from 30 GB of RAM and NVMe, with attention state partly in pinned RAM. Short chat decode is 13.2-15.7 tok/s for Q4 and 19.8-21.2 tok/s for Q3; at 155K context Q3 gives 348 tok/s prefill and 17.7 tok/s decode. A speculative block predicts the next layer's experts at 95-96% accuracy, with no token-level speculative decoding.

generation:

40.0 tokens/s

quant:

IQ3_S (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at ~40 tk/s on an RTX 4080 Super 16GB. Setup is the strata engine with an IQ3_S quant, run through a custom mini swe agent v2 with powershell, edit_file, view_image, read_file and search_files tools, plus guards for tool failures and auto-compaction. The one-prompt GTA-style three.js game took 3h 18m 6s and 22,845,556 tokens (22,533,061 input, 312,495 output). The user notes the IQ3 quant fucks up a lot.

generation:

59.0 tokens/s

quant:

2.469 bpw

kv:

FP8 E4M3

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B-Escha-W2 at about 59 t/s short context and about 50.5 t/s at 60K context on a 16 GB RTX 4080 Super, pushed to 98K context. Setup is Escha's SGLang build with a 2.469 bpw quant, FP8 KV cache and BF16 SSM state. MTP4 gave about 67.7 t/s at 64K context, but the user chose no speculation for maximum context. The user finds quality surprisingly good despite the aggressive quant.

generation:

39.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at ~39 tok/s fully GPU-offloaded on an RTX 4080 SUPER 16GB, with ~200-230 ms TTFT on short tasks. Setup is LM Studio on Windows 11 with the Ubuntu Codex CLI, on a 5800X3D with 3000mhz DDR4. On a real 26K-token workload the speed is ~34 tok/s. The user finds it surprisingly capable for local coding and agent work but too inconsistent to replace a hosted model, and an excellent local workhorse for bounded tasks.