Qwen2.5 32B Coder
on NVIDIA RTX 3090 · llama.cpp · 32,768 ctx
Use cases
coding
Summary
User reports Qwen2.5-Coder-32B at about 28 t/s generation and around 450 t/s prompt processing on a single RTX 3090.
Setup is llama.cpp build b4400 with a Q4_K_M quant at 32k context and a Q8 KV cache.
User calls it solid for autocomplete, but says it occasionally hallucinates imports in multi-file refactors.