Qwen3.8 27B
on NVIDIA RTX 5060 Ti 16GB · llama.cpp · 14,644 ctx
Oct 8, 2026
Summary
User reports Qwen3.8 27B at 28.5 t/s decode on a 14,644-token prompt on an RTX 5060 Ti 16GB, with prefill at 878-892 t/s.
Setup is an adaptive KV-streaming llama.cpp fork on one RTX 5060 Ti with GSQ-RCO IQ3_S weights, q8_0 KV cache, a 1536 MiB KV window and 131K context allocated, the whole model on the GPU, no speculative decoding; the host is a Ryzen 9600X with 32 GB single-channel DDR5.
Decode falls as the prompt grows: 25.2 t/s at 35K, 20.6 at 55K, 18.7 at 69K and 12.6 at 94K with the 1536 MiB window, and about 9 t/s near 131K. A smaller 512 MiB window drops faster (8.7 t/s at 94K). Prefill falls from 878-892 to 630-635 t/s. MTP is not usable with the fork; it gained about 5 t/s on stock llama.cpp.