llamaperf

Qwen3.8 27B

on NVIDIA RTX 2080 Ti 22GB (modded) · NInfer · 128,000 ctx

Tone: positiveNVIDIA hardware
Sep 7, 2026
Throughput
25.0 t/s gen
Quant
W8A16
KV cache
Q8
VRAM reported
22 GB

Summary

User reports Qwen3.8 27B at ~25 tok/s with standard autoregressive decoding (MTP0) on a modded RTX 2080 Ti 22GB. Setup is a Turing port of the NInfer engine with the official groupwise-int (W8A16) artifact and Q8 KV cache. With MTP3 speculative decoding (draft window = 3) the title gives 45 tok/s and the body ~456 tok/s, at ~65% acceptance rate. VRAM usage is ~17.5 GiB with the MTP draft weights loaded, leaving ~4.5-5.0 GiB free for KV cache.