llamaperf

Qwen3.8 27B

on 2× NVIDIA V100 16GB · llama.cpp

Tone: positiveNVIDIA hardware
Sep 19, 2026
Throughput
39.9 t/s gen · 1376.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q6_K_M (GGUF)
KV cache
Q8
System RAM
512 GB
VRAM reported
48 GB

Summary

User reports Qwen3.8 27B at 39.88 t/s decode and 1,376.87 t/s prompt processing on a mismatched pair of Tesla V100 PCIe cards (16 GB + 32 GB, 48 GB total). Setup is mainline llama.cpp with the Q6_K_M GGUF, tensor split, Flash Attention, Q8 KV cache, batch/ubatch 2,048, 20 threads, and NUMA distribute, with the 32 GB V100 as main GPU. Prompt processing measured 1,324.34 t/s at 4k and 1,221.49 t/s at 16k. A Qwen3.8 Flash Next 177B/6B-active MoE Q4_K_XL GGUF ran at 26.29 t/s decode and 380.67 t/s prompt on the same two cards.