llamaperf

Qwen3.8 27B RVN

on NVIDIA RTX 5090 · llama.cpp · 1,024 ctx

Tone: positiveNVIDIA hardware
Oct 26, 2026
Throughput
100.0 t/s gen · 1500-2500 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q6_K (GGUF)
KV cache
q4_0
System RAM
64 GB
VRAM reported
32 GB

Use cases

codingagenticvisionmultilinguallong-context

Summary

User reports Qwen3.8-27B RVN Q6_K at 100 tok/s at 1K context on a single RTX 5090 32 GB, up from 69 tok/s in LM Studio with the same GGUF and q4_0 KV cache. Setup is a native llama.cpp master build (CUDA 13.1, sm_120) with MTP speculation at 3 drafts, q4_0 K and V cache, 196608 configured context, and about 29.5 GB VRAM at 192K with vision. At ~60K context decode is 79-83 tok/s and at ~120K it is 61-66 tok/s; prefill is ~1500-2500 tok/s on both engines. With Chrome visible at 120K decode drops to 51 tok/s, 59 minimized and 64 closed. A 450 W power limit leaves decode within noise (78.6 vs 82.7 tok/s) but cuts prefill 11%. In a Codex CLI agentic run at 155-160K context the model wrote at a median 81 tok/s with peaks of 149.