Qwen3.8 27B
on NVIDIA RTX 4090 · llama.cpp · 8,192 ctx
Summary
User reports 47 t/s generation.
Source evidence (verbatim):
Source: https://computingforgeeks.com/run-qwen3-8-27b-locally/
# Run Qwen3.8-27B Locally: VRAM and Speed Tested
Hardware: RTX 4090 24 GB, Ubuntu 24.04, llama.cpp build b11200 (CUDA 12.8), Ollama 0.34.4. Model Qwen3.8-27B unsloth UD-Q4_K_XL, flash attention on, -ngl 99. Without MTP (plain run), 8K context, f16 KV: 47.0 tok/s generation, 17.0 GiB VRAM. 64K f16: 47.0 tok/s. 131K q8_0: 46.6 tok/s. 262K q4_0: 46.5 tok/s, 22.2 GiB. llama-bench generation without MTP: Q4_K_XL 47.9, Q5_K_XL 41.5, Q6_K 39.6 tok/s. llama-bench prompt processing: 2,600 to 3,000 tok/s. A 196,971-token prompt: prefill 121.5 s (1,621 tok/s), generation 26.6 tok/s. With MTP speculative decoding (draft-mtp, mtp-Qwen3.8-27B-Q4_0.gguf) at 8K context: Q4_K_XL 109.4 tok/s (runs 102.8, 109.4, 113.2), Q5_K_XL 95.8 tok/s, acceptance 69%.