Qwen3.8 27B
on NVIDIA RTX 5060 Ti 16GB · llama.cpp
Use cases
agentic
Summary
User reports Qwen3.8 27B at just over 50 t/s decode and just under 900 t/s prompt processing on a single 16GB RTX 5060 Ti.
Setup is llama.cpp with an IQ3_XXS GGUF pruned to 9.3 GB via vocabulary pruning, Q8_0 KV cache, MTP speculative decoding, and everything in VRAM.
The user gets just over 100k context with Q8 KV cache and a little over 170k with Q4 KV cache. They previously ran Gemma 4 26B A4B QAT at 90 t/s generation and just under 3000 t/s prompt processing, and Qwen 3.6 35B A3B, but found their intelligence insufficient for agentic use cases.