Qwen3.8 27B
2× AMD RX 7600 XT 16GB · llama.cpp · 35,000 ctx
- generation:
- 30.0 tokens/s
- prompt processing (prefill):
- 550.0 tokens/s
- quant:
- UD-Q4_K_XL (GGUF)
- kv:
- f16
User reports Qwen3.8 27B at 550 t/s prompt processing and 30 t/s generation on an RX 7600 XT 16GB plus RX 9060 XT 16GB, in a moderate context of 35k tokens. Setup is llama.cpp (latest git) with UD-Q4_K_XL GGUF weights, f16 KV cache, tensor split 60/40, flash attention on, and draft-MTP plus ngram-mod speculative decoding. ROCm was considerably faster than Vulkan. Adaptive KV streaming reached over 800 t/s prefill but limited the user to lower quantizations and left the second card unused. The user asks whether Strata or another fork would do better on this 32GB RAM system.