llamaperf

NVIDIA RTX 2000 Ada reports

2 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX 2000 Ada speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Qwen3.5 35B (3B active)

2× NVIDIA RTX 2000 Ada · Ollama · 16,000 ctx

Tone: positiveNVIDIA hardware
generation:
58.9 tokens/s
prompt processing (prefill):
2317.5 tokens/s
quant:
Q4_K_M (GGUF)
agenticcodingtool-uselong-context

User benchmarks a broad set of local models on a dual RTX 2000 Ada 16GB setup (32GB total VRAM) with 64GB system RAM, running Ollama in Docker on Debian 13. The headline result is Qwen3.5 35B-A3B Q4_K_M at 58.9 tok/s generation and 2317.5 tok/s prefill on a 16K-token prompt, though the user notes a reproducible CUDA illegal-memory-access crash with Flash Attention enabled on one prompt shape. Setup is Ollama with GGUF quants, 16K context for the headline run, and no KV cache quant stated. The user also reports a 100-task quality harness (250 points) and a separate speed harness with 512, 2K and 16K prompts. Other models tested include Gemma 4 26B-A4B Q4 at 42.5 tok/s, GPT-OSS 20B at 46.9 tok/s, Qwen3 30B-A3B IQ3 at 53.4 tok/s, and LFM2 24B-A2B Q4 at 102 tok/s. Dense models were slower: Mistral Small 3.2 24B Q4 at 12.6 tok/s and Qwen3.8 27B Q6_K_M at 8.8 tok/s. The user kept seven models and deleted the rest.

Oct 11, 2026

Gemma 4

2× NVIDIA RTX 2000 Ada · TensorSharp

Tone: positiveNVIDIA hardware
generation:
51.7 tokens/s
prompt processing (prefill):
2488.0 tokens/s
quant:
Q8_0

User benchmarks TensorSharp multi-GPU tensor parallelism on 2x RTX 2000 Ada 16GB. Gemma 4 E4B Q8_0 is the primary model among several tested. TP=2 raises decode speed from 37.3 to 51.7 tok/s.

Sep 7, 2026