Qwen3.8 27B
on NVIDIA RTX 3090 · llama.cpp · 16,384 ctx
Sep 18, 2026
Use cases
visioncodinglong-context
Summary
User reports Qwen3.8-27B (text + vision) as an 11.8 GB GGUF at 3.47 bpw, running at 42.3 tok/s single stream on an RTX 3090, matching Q4_K_M on the same card.
Setup is a patched llama.cpp build (13 files, 471 insertions on llama.cpp 6a1a922d) with the Opti-27B GGUF and mmproj, 16,384 context, full GPU offload. Prefill at 7k tokens is 1,127 tok/s; three concurrent streams give about 24 tok/s each.
Wikitext-2 perplexity is 6.487 versus 6.456 for FP16 and 6.457 for Q4_K_M at 16.8 GB. Four 16k conversations with vision fit in 17 GB; one 65k conversation with vision fits in 15 GB. MMLU-Pro is 53.9 +/- 0.5.