Qwen3 14B
on 2× NVIDIA RTX 2080 Ti · llama.cpp · 32,768 ctx
Use cases
codingtool-usevisionagentic
Summary
User reports Qwen3-14B at 68.1 t/s generation and 3057 t/s prompt processing on a 2x RTX 2080 Ti node.
Setup is llama.cpp with Q4_K_M quant, 32768 context, and a Qwen3-0.6B-Q8_0 draft model for speculative decoding with depth 6.
The user also reports Pixtral-12B at 77.0 t/s generation and 2862 t/s prompt processing, and a distributed 4-stream aggregate of 118.68 t/s. They ask if Linux would give better t/s and mention P2P is working on Windows.