Qwen3.8 27B
on NVIDIA V100 32GB · llama.cpp
Oct 8, 2026
Summary
User reports Qwen3.8 27B at about 50 t/s decode on a V100 32GB.
Setup is llama.cpp with UD-Q4-K-M (MTP) and batch size 4096, ubatch 2048, on a PG500-216 card at 185W with a custom compile.
User notes prefill drops off quickly and asks for pointers on improving it.