Qwen3.8 27B
on NVIDIA V100 32GB · llama.cpp · 131,072 ctx
Sep 29, 2026
Summary
User reports Qwen3.8-27B at 33.41 tok/s generation and 826.03 tok/s prefill on a single Tesla V100 32GB.
Setup is llama.cpp with UD-Q4_K_XL GGUF at 131,072 context; generation falls to 27.95 tok/s at 32k depth and 15.84 tok/s at 131k depth. Peak VRAM was 25,264 MiB of 32,768 MiB.
KV cache quantization (-ctk q8_0 -ctv q8_0) silently fell back to CPU with 0% GPU utilization; CUDA 13 dropped Volta support.