llamaperf

Qwen3.8 27B

on AMD RX 9070 XT 16GB · llama.cpp · 262,144 ctx

Tone: positiveAMD hardware
Oct 17, 2026
Throughput
36.5 t/s gen · 1368.4 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ4_XS (GGUF)
KV cache
Q8_0/Q4_0
VRAM reported
16 GB

Summary

User reports Qwen3.8-27B at 36.5 t/s decode on a single 16 GB RX 9070 XT, using a custom llama.cpp fork with adaptive KV streaming over PCIe. Setup is llama.cpp with IQ4_XS weights and Q8_0/Q4_0 KV cache, running the full 262,144-token context with the KV cache held in host RAM (~7.4 GB) and streamed in bounded working sets. Decode falls from 36.5 t/s at 1k to 9.5 t/s at 262k; with MTP speculative decoding it reaches 71.8-86.5 t/s at 1k-4k and 31.1 t/s at 262k. Prefill is 1368.4 t/s at 1k and 585.8 t/s at 262k. The card runs at a 231W limit with a 3.6% prefill penalty and no decode loss.