Qwen3.8 27B
on 2× AMD RX 6800 XT 16GB · llama.cpp · 65,536 ctx
Summary
User reports Qwen3.8-27B at 18.5-19.4 tok/s decode on two RX 6800 XT/6800 16GB cards with layer split.
Setup is llama.cpp via Ollama 0.33.2 with Q4_K_M, 65536 context, MTP speculative decoding (draft_num_predict 2). Prefill is ~230 tok/s on a fresh 15k-token prompt; cached runs showed 3,600-13,000 tok/s. Forced flash-attn dropped decode to 16.4 tok/s.
User asks whether ~58% of theoretical bandwidth (297 GB/s of 512 GB/s) is the expected ceiling for hybrid linear-attention models on ROCm, and notes Vulkan crashes mid-decode with DeviceLostError.