llamaperf

Qwen3.8 27B

on NVIDIA V100 32GB · llama.cpp · 1,024 ctx

NVIDIA hardware
Oct 9, 2026
Throughput
65.1 t/s gen
Quant
Q4_K_M (GGUF)
VRAM reported
32 GB

Use cases

codingagentictool-use

Summary

User reports Qwen3.8-27B at 65.07 tok/s on a single Tesla V100-SXM2-32GB at 1K context. Setup is llama.cpp (pinned tag b10793) with Q4_K_M GGUF and MTP speculative decoding enabled, 128K context window, single stream. Decode falls to 62.41 tok/s at 8K, 61.24 at 32K, 52.51 at 64K and 42.22 at 128K. With MTP off on a T2-001 packed-attention build the same card gives 35.22 tok/s at 1K and 23.83 at 128K; stock llama.cpp gives 35.95 at 1K and 16.49 at 128K. Prompt caching cuts multi-turn wall time from 22.94 s to 3.08 s per turn.