RTX 5080

reported speed:

12.0 tokens/s generation

quant:

NVFP4 (GGUF)

kv:

q4_0

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Qwen3.8 27B at around 12 t/s on an RTX 5080 16GB at 49,152 context. Setup is llama.cpp with an NVFP4 GGUF, q4_0 K cache and iq4_nl V cache, flash attention on, 512 batch and ubatch, 16 threads. User is asking for pi.dev plugin suggestions for context, compaction and memory management, and is disappointed that compaction settings are kept separate from model settings.

reported speed:

60.0 tokens/s generation · 1000.0 tokens/s prompt processing

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports prefill around 1000 PP/s and decode 50-70 t/s with Qwen 3.6 35B-A3B on an RTX 5080 eGPU. User asks about CPU instruction set differences for offloading.

reported speed:

6.0 tokens/s generation

quant:

IQ3_XXS (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 6 t/s on an RTX 5080 16GB with 64GB RAM, and finds it unusable. Setup is llama.cpp with offloading to CPU/RAM. The first attempt used the IQ3_XXS quant, the second the Q4_K_M quant at 4.27bpw AD.

reported speed:

75.0 tokens/s generation

quant:

IQ4_XS (gguf)

kv:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports hybrid quantization to fit MTP and long context in 16 GB VRAM. Setup uses MTP draft with spec-type draft-mtp and spec-draft-n-max 3, at 85,000 context, with q4_0 KV cache quantization for both cache types.

reported speed:

13.2 tokens/s generation

quant:

Q4_K_M (gguf)

kv:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a first tuning attempt with selective FFN offload to CPU improving speed over whole-layer offload. True Q4_K_M runs at 13.2 tok/s near 50-61K context. MTP is worse at deep context. Coding quality improved over IQ3.

quant:

IQ4_XS (gguf)

User benchmarks 21 Qwen3.8 27B variants on 16 GB of VRAM, sorted by mean KLD. The best overall is bartowski/Qwen3.8-27B-IQ4_XS. The best uncensored is huihui-ai/Huihui-Qwen3.8-27B-abliterated-UD-IQ4_XS. Also mentioned are jpetrina/Qwen3.8-27B-IQ4_XS-pure and Bucoid/Qwen3.8-27B-Uncensored-IQ4_XS_4BPW.

reported speed:

56.0 tokens/s generation · 1584.0 tokens/s prompt processing

quant:

Q4_K_XL (gguf)

kv:

Q8

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a 35B Q4_K_XL model at 128k context runs best with no MTP and --fit-target 1536. MTP does not help at 128k context. A 27B IQ3 model fits fully on GPU and benefits from MTP at 73 tok/s.