Qwen3.8 27B
on NVIDIA V100 32GB · llama.cpp · 128,000 ctx
Sep 29, 2026
Use cases
agentic
Summary
User reports Qwen3.8 27B at 65.07 t/s decode at 1K context on a Tesla V100-SXM2-32GB.
Setup is llama.cpp pinned at tag b10793 with the repo's patches, Q4_K_M GGUF, 1 parallel slot, multi-token prediction speculative decoding enabled.
Decode falls to 42.22 t/s at 128K context; stock llama.cpp gives 35.95 t/s at 1K and 16.49 t/s at 128K, and the patched build without MTP gives 35.22 t/s at 1K and 23.83 t/s at 128K. Multi-turn agentic sessions with prompt caching take 3.08 s per turn versus 22.94 s without.