Qwen3.8 27B
on NVIDIA V100 32GB · llama.cpp · 262,000 ctx
Oct 9, 2026
Summary
User reports Qwen3.8 27B UD-Q4_K_M at 57.25 t/s decode (depth 0) on a single Tesla V100-PCIE-32GB.
Setup is llama.cpp with llama-split-bench --profile, MTP enabled, measured up to 262k context. Prefill is about 720 t/s on new prompts.
Decode falls to 20.49 t/s at depth ~260k. An initial run suffered thermal throttling (15.68 t/s at 64k depth) due to BMC fan control; values shown are with fan mode set to Full.