llamaperf

Qwen3.8 27B

on NVIDIA V100 32GB · llama.cpp

Tone: positiveNVIDIA hardware
Sep 18, 2026
Throughput
23.2 t/s gen · 433.1 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Summary

User reports Qwen3.8 27B on a V100 at 433.1 t/s prompt processing and 23.19 t/s generation with a Volta-optimized llama.cpp fork. Benchmark uses 100k cached tokens plus 1k new prompt tokens and 64 generated tokens; the fork improves the FlashAttention path for roughly 40% higher prompt processing. Upstream llama.cpp gave 306.7 t/s prompt and 23.29 t/s generation on the same setup. The user notes the gain is mostly limited to Qwen-like model shapes and also tested Qwen3.5 122B A10B, Laguna-S-2.1, and GLM-5.2.