NVIDIA T1000 8GB

NVIDIA · 8GB · 2 reports

On the NVIDIA T1000 8GB, Ling-3.0 7.9B · 1.3B active at 4-bit, the model most run on it, generates a median of 54.0 tokens per second from one plain run (community reports, as of 25 Oct 2026).

As of 25 Oct 2026, the models most run on the NVIDIA T1000 8GB, with generation and prompt-processing medians from plain runs (one device, one request, no speculative decoding):

Ling-3.0 7.9B · 1.3B active
54.0gen t/s at 4-bit · one run
PP 245.0 t/s · 1 run

Engines people use on it: llama.cpp 2 (reports)

See reported performance Estimate what fits

How to compare these reports →Filter reports by setup →Best local LLMs for 8 GB of VRAM →

Run models on your NVIDIA T1000 8GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Benchmark with your agent Paste a result

Reported performance on the NVIDIA T1000 8GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 8 GB of VRAM.

This page is thin (2 of 3 reports needed for indexing). Help fill it in.

Date ↓Date ↑Token speed ↓Token speed ↑Prefill ↓Prefill ↑Context ↓Context ↑Ranked

Ling-3.0 7.9B (1.3B active) Tiny

NVIDIA T1000 8GB · llama.cpp · 16,384 ctx

generation:: 54.0 tokens/s
prompt processing (prefill):: 245.0 tokens/s
quant:: Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

tool-uselong-context

User benchmarks Ling 3.0 tiny against Gemma 4 26B-A4B for batched JSON speaker attribution on an NVIDIA T1000 8GB. Ling 3.0 tiny runs at 245 t/s prefill and 54 t/s decode, 1.8x faster end to end than Gemma. Setup is llama.cpp with the official Q4_K_M GGUF at 16384 context, all 25 layers on the GPU, no speculative decoding. Gemma uses UD-Q4_K_XL with experts in system RAM and an MTP draft head. Ling scores 56.9% exact speaker match versus Gemma's 94.6% across 485 lines, with narrator swaps and refusals. User is not adopting it.

Bonsai 2 27B

NVIDIA T1000 8GB · llama.cpp · 8,192 ctx

generation:: 6.0 tokens/s
prompt processing (prefill):: 18.0 tokens/s
quant:: PTQ1_0 (GGUF)
kv:: q8_0

summarizationlong-context

User benchmarks Ternary Bonsai 2 27B at ~6 t/s generation and ~18 t/s prefill on an NVIDIA T1000 8GB, comparing it against Gemma 4 26B-A4B for audiobook speaker attribution. Setup is a PrismML llama.cpp fork with PTQ1_0 ternary weights (5.95 GB), q8_0 KV cache, 8192 context, and all layers on GPU; the model scores 448/480 (93.3%) exact speaker matches on four chapters. The user does not adopt it, citing narrator identity drift, adjacent-turn swaps, dropped lines, and an intonation regression at 5-10x the wall-clock of the incumbent Gemma 4 26B-A4B MoE.

Get a weekly email of new NVIDIA T1000 8GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardware	Quant / engine	Reported context	Generation
Ling-3.0 7.9B (1.3B active) Tiny NVIDIA T1000 8GB	Q4_K_M llama.cpp	16,384	54.0 tokens/s
Bonsai 2 27B NVIDIA T1000 8GB	PTQ1_0 llama.cpp	8,192	6.0 tokens/s

Frequently asked

How fast is the NVIDIA T1000 8GB for local LLMs?

On the NVIDIA T1000 8GB, Ling-3.0 7.9B · 1.3B active at 4-bit, the model most run on it, generates a median of 54.0 tokens per second from one plain run (community reports, as of 25 Oct 2026).