Qwen3.8 27B
on Unknown GPU · llama.cpp · 110,000 ctx
Use cases
long-context
Summary
User reports Qwen3.8 27B at 25 tok/s generation with 110K context on a 12GB VRAM, 16GB RAM machine.
Setup is llama.cpp with an IQ3_XXS GGUF quant, q4_0 KV cache, flash attention on, -ngl 62 and 110000 context; prefill runs 750 to 1200 tok/s.
Other context points: 180K at 8 tok/s, 150K at 11 tok/s, 120K at 17 tok/s, 90K at 35 tok/s and 60K at 36 tok/s. The gains came from dropping the display from 4K to 1080p and turning off Moonlight streaming to free VRAM. MTP took too much VRAM to be usable.