llamaperf

Qwen3.8 27B

on 2× NVIDIA Tesla P100 16GB · llama.cpp

NVIDIA hardware
Oct 9, 2026
Throughput
24.5 t/s gen
Quant
Q4_K_XL (GGUF)
KV cache
F16

Summary

User benchmarks llama.cpp CUDA patches on two Tesla P100-PCIE-16GB cards, reporting Qwen3.8-27B Q4_K_XL at 24.5 tok/s decode with -sm tensor, up from 22.3 tok/s stock. Setup is llama.cpp b10660 with patches 0001-0012, CUDA 12.4, F16 KV cache, tensor split across two cards. The README also reports Llama-3.1-8B Q4_K_M at 79.57 tok/s on one card (2.03x stock), Qwen3.8-27B pp8192 at 378.7 tok/s, and a real 74,919-token request at 20.92 tok/s with F16 KV versus 16.85 with q8_0. Four concurrent agents reach 20.3 tok/s each (72.9 aggregate).