llamaperf

Qwen3.6 35B (3B active)

on NVIDIA RTX 3080 Ti · llama.cpp · 131,072 ctx

NVIDIA hardware
Nov 19, 2026
Throughput
50.5 t/s gen · 303.7 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
Q4_K_M (GGUF)
KV cache
q8_0 keys / q4_0 values
System RAM
32 GB
VRAM reported
12 GB

Use cases

tool-uselong-context

Summary

User reports Qwen3.6-35B-A3B at 50.47 tok/s generation and 303.7 tok/s prompt evaluation on an RTX 3080 Ti 12 GB, with a 131072-token context. Setup is llama.cpp b12530 with Q4_K_M GGUF, q8_0 keys and q4_0 values KV cache, Flash Attention on, one slot, and --n-cpu-moe 22 offloading experts to the 32 GB host RAM. The server loaded in 12.6 s and used 11,817 MiB of GPU memory, leaving 270 MiB free; 4.9 GB system RAM remained available. Both OpenAI- and Anthropic-format tool calls passed.