Qwen3.6 35B (3B active)
on Tesla K80 · Pantheon
Use cases
codingcreative-writingsummarization
Summary
User reports Qwen3.6-35B-A3B at about 21 tok/s decode on a single Tesla K80 (both dies), in a 2-core, 8 GB, no-AVX host.
Setup is the Pantheon engine on the Foundry C library with Q3_K_M weights held on the card rather than in system RAM; the CUDA backend targets Kepler sm_37.
Prompt reading is about 110-120 tok/s up to roughly 8k tokens, about 75 at 14k and about 55 at 30k. Prefix caching is built in, and an opencode second turn with a 14k-token system prompt dropped from 170 s to 9 s. The user notes prefill is the K80's weak spot. Nex-N2-mini also ran at about 21 tok/s decode.