Qwen3.8 27B
on 2× NVIDIA V100 16GB · llama.cpp
Sep 19, 2026
Summary
User reports Qwen3.8 27B at 39.88 t/s decode and 1,376.87 t/s prompt processing on a mismatched pair of Tesla V100 PCIe cards (16 GB + 32 GB, 48 GB total).
Setup is mainline llama.cpp with the Q6_K_M GGUF, tensor split, Flash Attention, Q8 KV cache, batch/ubatch 2,048, 20 threads, and NUMA distribute, with the 32 GB V100 as main GPU.
Prompt processing measured 1,324.34 t/s at 4k and 1,221.49 t/s at 16k. A Qwen3.8 Flash Next 177B/6B-active MoE Q4_K_XL GGUF ran at 26.29 t/s decode and 380.67 t/s prompt on the same two cards.