llamaperf

Qwen3.6 35B (3B active)

on NVIDIA Tesla P40 24GB · llama.cpp · 131,072 ctx

Tone: positiveNVIDIA hardware
Sep 25, 2026
Throughput
51.6 t/s gen · 982.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q3_K_XL (GGUF)
KV cache
Q8_0
VRAM reported
24 GB

Use cases

long-context

Summary

User reports Qwen3.6-35B-A3B at 51.6 tok/s on a Tesla P40 24GB under WSL2. Setup is llama.cpp built for sm_61 with UD-Q3_K_XL (16.85 GB, 17.23 GB with MTP), full GPU offload, MTP speculative decoding at --spec-draft-n-max 2, and Q8_0 KV cache at 131072 context. Baseline without speculation is 37.9 tok/s; MTP with speculation disabled lands at 38.3 tok/s, confirming the speedup is real. Prefill is 982 tok/s (pp512). Context scaling from 4096 to 131072 costs about 1 GB VRAM and about 2% throughput. Needle-in-a-haystack passed 9 of 9 at 10/50/90 percent depth, with 6315-token prompts at about 790 tok/s prefill and 25111-token prompts at 422-609 tok/s prefill. All generation figures are single-stream.