Qwen3.8 27B
on 2× NVIDIA T4 16GB · ExLlamaV3
Use cases
codingcreative-writing
Summary
User benchmarks Qwen3.8-27B on two free Kaggle T4s (no P2P, 70W cap per card) with EXL3 4.0bpw under exllamav3, tensor parallel over PCIe, and a DFlash2 block drafter of 7 tokens.
Overall throughput is 55-57 tok/s with drafting versus ~18-19 without; HumanEval reaches 154 tok/s and prose 35-37 tok/s.
Prose acceptance is the limit at about 2.5 accepted tokens per step versus ~7 on code. On the same cards llama.cpp best was 16.5 tok/s and vLLM int4 with its own MTP drafter did 40.8 tok/s.