Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
visionagenticlong-context
User reports Qwen3.8-27B at 71.57 t/s decode on a single RTX 5080 16GB at 131,072 context.
Setup is NInfer v1.3 with mixed Q3/Q4/Q5 groupwise quantization (~3.95 BPW), Q4 group64 KV cache, MTP-3 speculative decoding, and Vision enabled. The 118,001-token prompt prefilled at 1380.61 t/s with 44.74% MTP acceptance.
A short 512-token request averaged 84.7 t/s decode, and a video test reached 97.1 t/s decode with 100% MTP acceptance.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at around 100 t/s prefill and 18-20 t/s decode on 2x RTX 5080 with 128 GB DDR5.
Setup is llama.cpp with the unsloth UD-Q5_K_XL GGUF, a Q8_0 KV cache, 262,144 context, MTP speculative decoding with a Q4_K_M draft model, and ncmoe42 offloading 42 MoE layers to CPU.
User asks whether switching to vLLM or SGLang would give better performance, noting the scarcity of NVFP4/FP8 GGUF files.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 27B at around 12 t/s on an RTX 5080 16GB at 49,152 context.
Setup is llama.cpp with an NVFP4 GGUF, q4_0 K cache and iq4_nl V cache, flash attention on, 512 batch and ubatch, 16 threads.
User is asking for pi.dev plugin suggestions for context, compaction and memory management, and is disappointed that compaction settings are kept separate from model settings.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports prefill around 1000 PP/s and decode 50-70 t/s with Qwen 3.6 35B-A3B on an RTX 5080 eGPU.
User asks about CPU instruction set differences for offloading.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8-Flash-Next at 6 t/s on an RTX 5080 16GB with 64GB RAM, and finds it unusable.
Setup is llama.cpp with offloading to CPU/RAM. The first attempt used the IQ3_XXS quant, the second the Q4_K_M quant at 4.27bpw AD.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports hybrid quantization to fit MTP and long context in 16 GB VRAM.
Setup uses MTP draft with spec-type draft-mtp and spec-draft-n-max 3, at 85,000 context, with q4_0 KV cache quantization for both cache types.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports a first tuning attempt with selective FFN offload to CPU improving speed over whole-layer offload.
True Q4_K_M runs at 13.2 tok/s near 50-61K context.
MTP is worse at deep context. Coding quality improved over IQ3.
User benchmarks 21 Qwen3.8 27B variants on 16 GB of VRAM, sorted by mean KLD.
The best overall is bartowski/Qwen3.8-27B-IQ4_XS. The best uncensored is huihui-ai/Huihui-Qwen3.8-27B-abliterated-UD-IQ4_XS.
Also mentioned are jpetrina/Qwen3.8-27B-IQ4_XS-pure and Bucoid/Qwen3.8-27B-Uncensored-IQ4_XS_4BPW.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports a 35B Q4_K_XL model at 128k context runs best with no MTP and --fit-target 1536.
MTP does not help at 128k context.
A 27B IQ3 model fits fully on GPU and benefits from MTP at 73 tok/s.