Qwen3.8 27B
on 2× NVIDIA RTX 4000 Ada · llama.cpp · 32,768 ctx
Use cases
coding
Summary
User reports Qwen3.8-27B at 29.39 t/s generation and 350.96 t/s prompt processing on two RTX 4000 Ada GPUs (20GB each).
Setup is llama.cpp (llama-server) with Q8_0 GGUF weights, q8_0 KV cache, 32768 context, flash attention on, unified KV cache, and an MTP draft model plus vision projector, all layers offloaded to the GPUs.
The figures come from a first prompt of 376 tokens (1.1s) and 109 generated tokens (3.7s). The user asks for tuning advice and for help wiring Claude Code and Cline in VS Code to the local server.