llamaperf

Qwen3.6 35B (3B active)

on Tesla K80 · Pantheon

Tone: positive
Oct 20, 2026
Throughput
21.0 t/s gen · 110-120 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q3_K_M (GGUF)
System RAM
8 GB

Use cases

codingcreative-writingsummarization

Summary

User reports Qwen3.6-35B-A3B at about 21 tok/s decode on a single Tesla K80 (both dies), in a 2-core, 8 GB, no-AVX host. Setup is the Pantheon engine on the Foundry C library with Q3_K_M weights held on the card rather than in system RAM; the CUDA backend targets Kepler sm_37. Prompt reading is about 110-120 tok/s up to roughly 8k tokens, about 75 at 14k and about 55 at 30k. Prefix caching is built in, and an opencode second turn with a 14k-token system prompt dropped from 170 s to 9 s. The user notes prefill is the K80's weak spot. Nex-N2-mini also ran at about 21 tok/s decode.