llamaperf

Qwen3.8 27B

on NVIDIA RTX 3090 · llama.cpp · 16,384 ctx

Tone: positiveNVIDIA hardware
Sep 18, 2026
Throughput
42.3 t/s gen · 1127.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
3.47 bpw (GGUF)
VRAM reported
24 GB

Use cases

visioncodinglong-context

Summary

User reports Qwen3.8-27B (text + vision) as an 11.8 GB GGUF at 3.47 bpw, running at 42.3 tok/s single stream on an RTX 3090, matching Q4_K_M on the same card. Setup is a patched llama.cpp build (13 files, 471 insertions on llama.cpp 6a1a922d) with the Opti-27B GGUF and mmproj, 16,384 context, full GPU offload. Prefill at 7k tokens is 1,127 tok/s; three concurrent streams give about 24 tok/s each. Wikitext-2 perplexity is 6.487 versus 6.456 for FP16 and 6.457 for Q4_K_M at 16.8 GB. Four 16k conversations with vision fit in 17 GB; one 65k conversation with vision fits in 15 GB. MMLU-Pro is 53.9 +/- 0.5.