Qwen3.8 27B
on AMD RX 9070 XT 16GB · llama.cpp · 262,144 ctx
Summary
User reports Qwen3.8-27B at 36.5 t/s decode on a single 16 GB RX 9070 XT, using a custom llama.cpp fork with adaptive KV streaming over PCIe.
Setup is llama.cpp with IQ4_XS weights and Q8_0/Q4_0 KV cache, running the full 262,144-token context with the KV cache held in host RAM (~7.4 GB) and streamed in bounded working sets.
Decode falls from 36.5 t/s at 1k to 9.5 t/s at 262k; with MTP speculative decoding it reaches 71.8-86.5 t/s at 1k-4k and 31.1 t/s at 262k. Prefill is 1368.4 t/s at 1k and 585.8 t/s at 262k. The card runs at a 231W limit with a 3.6% prefill penalty and no decode loss.