llamaperf

NVIDIA RTX 2000 Ada

NVIDIA · 16GB · 2 reports

Is this your card?

Engines people use on it: Ollama 1 (reports) · TensorSharp 1 (reports)

Run models on your NVIDIA RTX 2000 Ada? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA RTX 2000 Ada

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 16 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

2 reports on this card so far, so treat the figures as early. Add yours.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Qwen3.5 35B (3B active)

2× NVIDIA RTX 2000 Ada · Ollama · 16,000 ctx

Tone: positiveNVIDIA hardware
generation:
58.9 tokens/s
prompt processing (prefill):
2317.5 tokens/s
quant:
Q4_K_M (GGUF)
agenticcodingtool-uselong-context

User benchmarks a broad set of local models on a dual RTX 2000 Ada 16GB setup (32GB total VRAM) with 64GB system RAM, running Ollama in Docker on Debian 13. The headline result is Qwen3.5 35B-A3B Q4_K_M at 58.9 tok/s generation and 2317.5 tok/s prefill on a 16K-token prompt, though the user notes a reproducible CUDA illegal-memory-access crash with Flash Attention enabled on one prompt shape. Setup is Ollama with GGUF quants, 16K context for the headline run, and no KV cache quant stated. The user also reports a 100-task quality harness (250 points) and a separate speed harness with 512, 2K and 16K prompts. Other models tested include Gemma 4 26B-A4B Q4 at 42.5 tok/s, GPT-OSS 20B at 46.9 tok/s, Qwen3 30B-A3B IQ3 at 53.4 tok/s, and LFM2 24B-A2B Q4 at 102 tok/s. Dense models were slower: Mistral Small 3.2 24B Q4 at 12.6 tok/s and Qwen3.8 27B Q6_K_M at 8.8 tok/s. The user kept seven models and deleted the rest.

Oct 11, 2026

Gemma 4

2× NVIDIA RTX 2000 Ada · TensorSharp

Tone: positiveNVIDIA hardware
generation:
51.7 tokens/s
prompt processing (prefill):
2488.0 tokens/s
quant:
Q8_0

User benchmarks TensorSharp multi-GPU tensor parallelism on 2x RTX 2000 Ada 16GB. Gemma 4 E4B Q8_0 is the primary model among several tested. TP=2 raises decode speed from 37.3 to 51.7 tok/s.

Sep 7, 2026

Get a weekly email of new NVIDIA RTX 2000 Ada reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3.5 35B (3B active)
2× NVIDIA RTX 2000 Ada
Q4_K_M
Ollama
16,00058.9 tokens/s
Gemma 4
2× NVIDIA RTX 2000 Ada
Q8_0
TensorSharp
Not reported51.7 tokens/s