Qwen3.8 27B
on NVIDIA RTX 2080 Ti 22GB (modded) · NInfer · 128,000 ctx
Summary
User reports Qwen3.8 27B at ~25 tok/s with standard autoregressive decoding (MTP0) on a modded RTX 2080 Ti 22GB.
Setup is a Turing port of the NInfer engine with the official groupwise-int (W8A16) artifact and Q8 KV cache.
With MTP3 speculative decoding (draft window = 3) the title gives 45 tok/s and the body ~456 tok/s, at ~65% acceptance rate. VRAM usage is ~17.5 GiB with the MTP draft weights loaded, leaving ~4.5-5.0 GiB free for KV cache.