Qwen3.8 27B
on NVIDIA RTX 3090 · llama.cpp · 262,144 ctx
Use cases
long-context
Summary
User reports Qwen3.8-27B at 66 t/s decode at 262,144-token context depth on one RTX 3090, with the full context fitting in 23.5 GB.
Setup is a custom engine with unsloth UD-Q4_K_XL weights, K cache at 6 bits, V at 5 bits, and the newest 2,048 positions kept at q8_0, saving 1.1 GB versus a full q8_0 cache at 120k context.
KV quality was scored by KL divergence against llama.cpp q8_0 output: 0.00160 at 64k versus 0.00163 for q8_0 and 0.00161 for fp16, +8% at 32k, and the same 230/282 on the author's question set. The author notes Qwen3.8 is a hybrid with only 16 of 64 layers attention, so full-attention models may differ.