Qwen3.8 27B
on NVIDIA RTX 3090 · llama.cpp · 65,536 ctx
Use cases
coding
Summary
User reports Qwen3.8-27B Q4_K_M at a median of about 38 tokens/s on an RTX 3090 in a USB4 eGPU enclosure attached to a Beelink GTR7 mini PC.
Setup is llama.cpp with Q4_K_M weights, q8_0 KV cache, 64K context, flash attention on, and n-gram speculative decoding enabled.
Prompt evaluation runs about 900-1200 tokens/s on large prompts, with spikes up to about 280 t/s when speculation hits on repetitive code. VRAM use is 20.7 of 24.6 GB with the full 64K context.