Qwen3.8 27B
on Unknown GPU · llama.cpp-adaptive-kv-streaming · 262,144 ctx
Summary
User reports Qwen3.8 27B at 80 t/s generation and 1500 t/s prompt processing on a 16GB VRAM GPU.
Setup is llama.cpp-adaptive-kv-streaming with UD-IQ4_XS quant and q4_0 KV cache at 262k context, using a 1.5GB kv-stream-arena and batch/ubatch of 128.
At full 262k context the user gets around 20 t/s generation and 400 t/s prompt processing. The user notes the speed drop is linear and they normally compact at 180-200k. The fork was modified to support q4_0 KV, and the user mentions q8_0/q4_0 works with lower context or batch sizes.