llamaperf

Qwen3.8 27B

on 2× NVIDIA T4 16GB · ExLlamaV3

Tone: mixedNVIDIA hardware
Oct 25, 2026
Throughput
55-57 t/s gen
Quant
EXL3 4.0bpw (EXL3)

Use cases

codingcreative-writing

Summary

User benchmarks Qwen3.8-27B on two free Kaggle T4s (no P2P, 70W cap per card) with EXL3 4.0bpw under exllamav3, tensor parallel over PCIe, and a DFlash2 block drafter of 7 tokens. Overall throughput is 55-57 tok/s with drafting versus ~18-19 without; HumanEval reaches 154 tok/s and prose 35-37 tok/s. Prose acceptance is the limit at about 2.5 accepted tokens per step versus ~7 on code. On the same cards llama.cpp best was 16.5 tok/s and vLLM int4 with its own MTP drafter did 40.8 tok/s.