Qwen3.8 27B
on NVIDIA RTX 5060 Ti 16GB · llama-tierkv · 8,192 ctx
Use cases
coding
Summary
User reports Qwen3.8 27B at 63.7 tok/s decode on an RTX 5060 Ti 16GB, using MTP-5 speculative decoding at 8K context.
Setup is llama-tierkv (a llama.cpp fork) with IQ4_XS weights and a Q4_0 KV cache kept in three tiers (VRAM, RAM, SSD); the weights are on the GPU. Without speculation the same configuration decodes at 25.7 tok/s, and prefill is 680-850 tok/s.
At 256K context without speculation, decode is 22.70 tok/s.