Qwen3.8 27B
on NVIDIA RTX 5070 Ti · beellama.cpp · 96,256 ctx
Summary
User reports Qwen3.8-27B-UD-Q3_K_XL.gguf at up to 75 t/s tg and up to 1700 t/s pp on an RTX 5070 Ti 16GB, fully offloaded, with vision on CPU.
Setup is beellama.cpp with kvarn performance enhancements, MTP n-max=2, KV=kvarn3/kvarn3, MTP draft KV=kvarn2/kvarn2, 96256 context and 1024 tail tokens.
The user notes Q3_K_XL scores almost 100% on a needle test with kvarn3, while Q3_XXS does poorly at 84% with F16 KV and ~80% with Q3 KV, and that kvarn is better with KLD. The user advises not fearing Q3 quants or KV quantisation, and concludes that for 16GB VRAM Q3_K_XL with kvarn3 works well.