Qwen3.8 27B
NVIDIA RTX 4080 Super · llama.cpp · 73,728 ctx
- generation:
- 33.1 tokens/s
- quant:
- Q3_K_XL (GGUF)
- kv:
- Q8_0
long-context
User reports Qwen3.8-27B decode throughput collapsing from 33.1 t/s at 68K position to 1.4 t/s at 91K position on an RTX 4080 Super 16GB. Setup is llama.cpp (master 8144f31) with UD-Q3_K_XL GGUF and Q8_0 KV cache, no flash attention, 73728 context, single GPU with zero CPU tensor overrides. Prompt processing stays fast at ~1300 t/s at the same 91K position. The user notes the cliff is positional rather than a memory-spill issue, reproduced across q4_0/q5_0/q8_0 KV quants and with FA on/off, and also under Windows native b10488.