llamaperf

Qwen3.8 27B

on NVIDIA RTX 4090 · llama.cpp · 8,192 ctx

NVIDIA hardware
Oct 21, 2026
Throughput
47.0 t/s gen · 2600-3000 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
KV cache
F16

Summary

User reports 47 t/s generation. Source evidence (verbatim): Source: https://computingforgeeks.com/run-qwen3-8-27b-locally/ # Run Qwen3.8-27B Locally: VRAM and Speed Tested Hardware: RTX 4090 24 GB, Ubuntu 24.04, llama.cpp build b11200 (CUDA 12.8), Ollama 0.34.4. Model Qwen3.8-27B unsloth UD-Q4_K_XL, flash attention on, -ngl 99. Without MTP (plain run), 8K context, f16 KV: 47.0 tok/s generation, 17.0 GiB VRAM. 64K f16: 47.0 tok/s. 131K q8_0: 46.6 tok/s. 262K q4_0: 46.5 tok/s, 22.2 GiB. llama-bench generation without MTP: Q4_K_XL 47.9, Q5_K_XL 41.5, Q6_K 39.6 tok/s. llama-bench prompt processing: 2,600 to 3,000 tok/s. A 196,971-token prompt: prefill 121.5 s (1,621 tok/s), generation 26.6 tok/s. With MTP speculative decoding (draft-mtp, mtp-Qwen3.8-27B-Q4_0.gguf) at 8K context: Q4_K_XL 109.4 tok/s (runs 102.8, 109.4, 113.2), Q5_K_XL 95.8 tok/s, acceptance 69%.