Qwen3.8 27B
on AMD RX 9070 XT 16GB · ExLlamaV3 · 96,000 ctx
Use cases
codingcreative-writing
Summary
User reports Qwen3.8-27B at ~150 tok/s on code at short context on a single RX 9070 XT.
Setup is ExLlamaV3 with EXL3 3.0 bpw quant, DFlash2 speculative decoding, and TabbyAPI, running in a container with RDNA4 patches. Default context is 96k, with 128k available if nothing else uses the card.
At 120k context code generation drops to ~105 tok/s, prose to ~60 tok/s. Prompt processing is ~1,100-1,250 tok/s on short prompts and ~900 tok/s at 128k. Up to 3 concurrent requests share the context. GPQA Diamond scores 88.4% at xhigh effort and ~81% at medium.