Qwen3 14B
2× NVIDIA RTX 2080 Ti · llama.cpp · 32,768 ctx
- generation:
- 68.1 tokens/s
- prompt processing (prefill):
- 3057.0 tokens/s
- quant:
- Q4_K_M (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingtool-usevisionagentic
User reports Qwen3-14B at 68.1 t/s generation and 3057 t/s prompt processing on a 2x RTX 2080 Ti node. Setup is llama.cpp with Q4_K_M quant, 32768 context, and a Qwen3-0.6B-Q8_0 draft model for speculative decoding with depth 6. The user also reports Pixtral-12B at 77.0 t/s generation and 2862 t/s prompt processing, and a distributed 4-stream aggregate of 118.68 t/s. They ask if Linux would give better t/s and mention P2P is working on Windows.