llamaperf

Qwen3.8 27B

on NVIDIA V100 32GB · llama.cpp · 131,072 ctx

NVIDIA hardware
Oct 3, 2026
Throughput
47.2 t/s gen
Quant
IQ3_S (GGUF)
KV cache
q8_0
System RAM
251 GB
VRAM reported
32 GB

Summary

User reports Qwen3.8-27B at 47.2 t/s decode on one Tesla V100-SXM2-32GB (250 W) in an X99 box with 2x Xeon E5-2690 v4 and 251 GB RAM. Setup is llama.cpp (V100 tensor-parallel fork) with IQ3_S GGUF weights, q8_0 KV cache, 128k context, and MTP self-speculation with up to 3 draft tokens per step. A 64k context run with f16 KV reached 52.5 t/s with 91.9% draft acceptance; the 128k run had 88.0% acceptance. Per 8k-token stretches the rate ranged 46-66 t/s at 64k and 41-57 t/s at 128k.