Qwen3.8 27B
on 2× NVIDIA RTX 5060 Ti 16GB · ExLlamaV3 · 102,400 ctx
Use cases
coding
Summary
User reports Qwen3.8 27B at about 50 t/s on an RTX 5060 Ti 16GB plus RTX 3060 12GB, with MTP enabled the rate varies between 40 and 60 t/s.
Setup is ExLlamaV3 through TabbyAPI with the turboderp exl3-5.0bpw weights, 102,400 tokens of context, a Q8 KV cache and Q8 draft cache, tensor parallelism across the two cards, and 32GB of system RAM.
Without MTP the rate drops to 22 t/s. The user notes 1.3GB free on the 3060 and 0.6GB on the 5060 and is trying to rebalance the GPU split.