llamaperf

Qwen3.8 27B

on NVIDIA V100 32GB · llama.cpp · 128,000 ctx

NVIDIA hardware
Sep 29, 2026
Throughput
65.1 t/s gen
Quant
Q4_K_M (GGUF)
VRAM reported
32 GB

Use cases

agentic

Summary

User reports Qwen3.8 27B at 65.07 t/s decode at 1K context on a Tesla V100-SXM2-32GB. Setup is llama.cpp pinned at tag b10793 with the repo's patches, Q4_K_M GGUF, 1 parallel slot, multi-token prediction speculative decoding enabled. Decode falls to 42.22 t/s at 128K context; stock llama.cpp gives 35.95 t/s at 1K and 16.49 t/s at 128K, and the patched build without MTP gives 35.22 t/s at 1K and 23.83 t/s at 128K. Multi-turn agentic sessions with prompt caching take 3.08 s per turn versus 22.94 s without.