Qwen3.8 27B
NVIDIA RTX 5070 Ti Laptop 12GB · Unsloth Studio · 8,192 ctx
- generation:
- 4.5 tokens/s
- prompt processing (prefill):
- 20-27 tokens/s
- quant:
- UD-Q4_K_XL
- kv:
- Q8
- mtp (multi-token prediction):
- on
coding
User reports Qwen3.8-27B at around 4.5 tok/s on an RTX 5070 Ti Laptop GPU with 12GB VRAM and 32GB system RAM. Setup is Unsloth Studio with UD-Q4_K_XL (roughly 17.9GB), 8K context, Q8 KV cache, flash attention, one parallel slot and MTP speculative decoding with draft max 2. Around 10,978 MiB stays in VRAM and the rest is offloaded to the CPU. A longer response ran at 3.26 tok/s, a short factual one at 4.42 tok/s and a Python coding one at 4.53 tok/s, with prompt processing around 20-27 tok/s. Draft-token acceptance was around 78-83%.