llamaperf

NVIDIA Tesla P100 16GB

NVIDIA · 16GB · 17 reports

Is this your card?

On the NVIDIA Tesla P100 16GB, Qwen3.8 27B at 3-bit, the model most run on it, generates a median of 10.0 tokens per second from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the NVIDIA Tesla P100 16GB, each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

Bars are medians on one scale, 0 to 10.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: llama.cpp 9 (reports) · Strata 1 (reports)

Run models on your NVIDIA Tesla P100 16GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA Tesla P100 16GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 16 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Unknown family

2× NVIDIA Tesla P100 16GB

Tone: mixedNVIDIA hardware
generation:
38.0 tokens/s

User reports about 38 t/s average generation on 2x P100, with 60+ t/s on some tasks when MTP is enabled. Prefill is about 200 t/s, which the user calls painfully slow; the system draws 500W under load. The user says it works but is basically unusable for anything interactive and asks whether prefill can be sped up on this old hardware.

Nov 21, 2026

Qwen3.8 27B

2× NVIDIA Tesla P100 16GB · llama.cpp · 262,144 ctx

NVIDIA hardware
generation:
45.0 tokens/s
quant:
UD-Q5_K_XL (GGUF)
agentictool-uselong-contextmultilingual

User reports Qwen3.8-27B at about 45 tok/s in English and about 32 tok/s in Polish on 2x Tesla P100 16GB. Setup is llama.cpp with UD-Q5_K_XL at 262k context, row/tensor split with MTP, on an X79 platform (i7-3930K, ~44 GB DDR3) under Proxmox with GPU passthrough. The P100s serve as the main LLM backend for an agent with long prompts and many tool calls; a GTX 1080 handles vision, Whisper, embeddings and image generation. The user asks whether to replace the 1080 with a Titan V or 2080 Ti, and reports a Quadro GP100 was not detected at all on the board.

Nov 10, 2026
Tone: positiveNVIDIA hardware
generation:
30.0 tokens/s
agentictool-use

User reports about 30 tok/s with the Swift-1.5 version of Qwen3.8 Flash-Next on a Tesla P100 16GB in a PCIe3 x16 slot, after switching from llama.cpp to Strata. Setup is Strata with 64GB of ECC DDR4 at 2100 and a 131072-token context; the logs show expert cache hits of 63.5% to 77.8% with routed experts going over PCIe, so part of the model ran from system RAM. Individual runs in the pasted logs range from 28.7 to 33.3 tok/s on prompts of roughly 78k tokens, which were read in 2 to 3 s. The user says it serves their Hermes agent and calls it impeccable for a local LLM.

Oct 25, 2026
Tone: positiveNVIDIA hardware
generation:
33.3 tokens/s
prompt processing (prefill):
406.9 tokens/s
quant:
Q4_0 (GGUF)

User reports Qwen3.8-27B Q4_0 at 33.28 t/s decode and 406.9 t/s prefill on 2x Tesla P100 16GB with patched llama.cpp, up from 23.43 t/s decode and 233.6 t/s prefill on stock. Setup is llama.cpp v0.7.0 with 41 sm_60 patches, tensor split across two P100s, 5 interleaved rounds per config. The patch set also gives 77.6 t/s decode on Ornith-1.0-35B-A3B UD-Q4_K_M with tensor split, and layer split gives 20.68 t/s decode on the 27B. Tested only on 2x P100; some patches are generic for all cards.

Oct 16, 2026
Tone: mixedNVIDIA hardware
generation:
10.0 tokens/s
quant:
Q3_K_M (GGUF)
creative-writing

User reports Qwen3.8 27B at 10.01 t/s on a Tesla P100 16GB. Setup uses the Q3_K_M GGUF quant with a 2,180-token prompt that took 3 minutes 37 seconds. The user asks whether this speed is normal and whether a 14B model would be faster, noting the P100 is old hardware.

Oct 12, 2026
Tone: mixedNVIDIA hardware
generation:
5.8 tokens/s
prompt processing (prefill):
56.1 tokens/s
quant:
IQ3_XXS (GGUF)
codingagentictool-use

User reports Qwen3.8-Flash-Next at 5.84 t/s generation and 56.10 t/s prompt processing on an Nvidia P100 16GB. Setup is llama.cpp with IQ3_XXS quant and 80K context, running on an HP Z440 workstation with Intel Xeon E5-1650 v3 and 64GB RAM. The model is a 125B MoE with 6B active parameters, partially offloaded to CPU. User notes it is slow but works for non-time-critical tasks like email sorting and calendar management.

Oct 12, 2026
Tone: mixedNVIDIA hardware
generation:
25.0 tokens/s
quant:
Q6
coding

User reports Qwen3.8 35B at Q6 running at 25 t/s with about 128k context on 2x Tesla P100 16GB. User says the model cheats heavily on their personal benchmark (building a clock in Lua for Conky) and still has trouble getting it working. User asks whether their prompting is the problem or if they are jaded by frontier models, and whether to have Qwen push work to Opus/Sol.

Oct 12, 2026
Tone: positiveNVIDIA hardware
generation:
31.2 tokens/s
prompt processing (prefill):
326.0 tokens/s

User reports Qwen3.8 Flash-Next at 31.2 t/s decode on a single Tesla P100 16GB with PXA v3.1 Overdrive mode, using a 32 GB quant. Setup is PXA with a 32 GB quant, experts cached on card and rest streamed, single request. Overdrive off gives 20.0 t/s; two P100s reach 38.2 t/s, one V100 35.4 t/s, two V100s 55.7 t/s. Against Strata on the same P100, prose decode 31.1 vs 27.0 t/s, code 34.3 vs 34.1, prompt processing 4k 326 vs 226 t/s and 8k 338 vs 246 t/s.

Oct 11, 2026
NVIDIA hardware
generation:
24.5 tokens/s
quant:
Q4_K_XL (GGUF)
kv:
F16

User benchmarks llama.cpp CUDA patches on two Tesla P100-PCIE-16GB cards, reporting Qwen3.8-27B Q4_K_XL at 24.5 tok/s decode with -sm tensor, up from 22.3 tok/s stock. Setup is llama.cpp b10660 with patches 0001-0012, CUDA 12.4, F16 KV cache, tensor split across two cards. The README also reports Llama-3.1-8B Q4_K_M at 79.57 tok/s on one card (2.03x stock), Qwen3.8-27B pp8192 at 378.7 tok/s, and a real 74,919-token request at 20.92 tok/s with F16 KV versus 16.85 with q8_0. Four concurrent agents reach 20.3 tok/s each (72.9 aggregate).

Oct 9, 2026
Tone: positiveNVIDIA hardware
generation:
120-140 tokens/s
quant:
UD_Q4_K_XL (GGUF)
agentictool-use

User reports Qwen3.6 35B-A3B at 120-140 t/s generation on two Tesla P100s, with prefill around 1000 t/s at 0 context and 700 t/s at 13k context. Setup uses Unsloth's UD_Q4_K_XL quant; the engine is not named. The user praises the model for broad agentic tasks and notes it is fast enough to be conversational, but criticizes it for coding, persona holding, and hallucination. They also mention running Qwen3.8 Flash-Next at 30-40 t/s with Strata at Q3.

Oct 9, 2026
NVIDIA hardware
generation:
32.6 tokens/s
prompt processing (prefill):
493.0 tokens/s
quant:
Q6_K (GGUF)
kv:
q4_0

User reports Qwen3.8-27B Q6_K at 32.6 t/s decode (tg256, no MTP) on 2x Tesla P100 16GB with tensor split, versus 17.51 t/s upstream at the fork point. Setup is a llama.cpp fork with CUDA work for Pascal (sm_60), q4_0 KV cache, and fp16 math with fp32 accumulation. Prefill pp2048 at 0 context is 493 t/s versus ~250 t/s upstream. With MTP speculative decoding the fork reaches 54 t/s at 2k context and 29-35 t/s at 260k context. Prefill at 260k context is 123 t/s filling and 153 t/s for a question on a loaded context. Perplexity on the gate corpus at -c 4096 is 2.6101.

Oct 6, 2026
Tone: positiveNVIDIA hardware
generation:
110.0 tokens/s
quant:
Q4_0 (GGUF)
kv:
F16
codingagenticlong-context

User reports Nemotron-3.5-Lightning-30B-A3B at 110 tok/s writing code on two Tesla P100 16GB cards. Setup is llama.cpp b10970 with Q4_0 weights, F16 KV cache, tensor split across both cards, and the model's built-in MTP draft head at n-max 2. The same model reaches 95 tok/s on prose, 50 tok/s at 128k context, 39 tok/s at 256k, and 16 tok/s at 1M tokens. The study covers 545 speed measurements of 29 models from 2B to 122B parameters, all weights and KV cache in VRAM with no system RAM offload. A 119B MoE model runs 39 tok/s against 4.3 tok/s for a 70B dense model on the same cards. Tensor split makes dense models from 8B up 21-44% faster. A single P100 throttles to 906 MHz and loses 25% under sustained load, while two cards share the heat and lose 5.5%.

Oct 6, 2026

Qwen3.8 27B

2× NVIDIA Tesla P100 16GB · llama.cpp · 262,144 ctx

Tone: positiveNVIDIA hardware
generation:
50-60 tokens/s
prompt processing (prefill):
350.0 tokens/s
quant:
Q6_K (GGUF)
codingcreative-writing

User reports Qwen3.8 27B at 50-60 t/s generation and 350 t/s prefill on 2x Tesla P100 16GB with a custom llama.cpp fork. Setup is llama.cpp with Q6_K quant, 262144 context, batch 32768, ubatch 1024, and MTP speculative decoding. At 260k context, generation is 30-35 t/s and prefill is 110 t/s. GPUs are capped at 175W/250W each and run at 79C with minor thermal throttling; user estimates 5-10% higher numbers with better cooling.

Sep 27, 2026
Tone: positiveNVIDIA hardware
generation:
54-60 tokens/s
prompt processing (prefill):
440-500 tokens/s
quant:
UD_Q4_K_XL (GGUF)
kv:
16bit
codingcreative-writing

User reports Qwen3.6 35B A3B at 54-60 t/s generation (66-72 t/s on code) on a single Tesla P100 16GB, up from 30-35 t/s prose on an RX 6600 XT. Setup is llama.cpp with shinbunbun patches, UD_Q4_K_XL quant, 16-bit KV cache, MTP speculative decoding, and --n-cpu-moe 22, running 32k context. Prefill is 600 t/s at 0 ctx dropping to 440-500 t/s by 10k. The user notes the P100 is underrated for the price and has a second card coming for full offload.

Sep 27, 2026
Tone: mixedNVIDIA hardware
generation:
85.0 tokens/s
prompt processing (prefill):
300.0 tokens/s
quant:
Q4_K_XL (GGUF)
agentic

User reports Qwen3.6-35B-A3B at ~85 t/s generation and ~300 t/s prompt processing on 2x Tesla P100. Setup is llama.cpp with the Q4_K_XL GGUF, MTP speculative decoding, and community P100 patches that added about 50% decode; a single P100 on Q2_K_XL reached ~76 t/s generation and ~250 t/s prompt processing. Card count barely affects single-stream speed, and PCIe lane width (x16/x16, x16/x8, x8/x8) made no difference. The user corrects an earlier 3-card result that was invalidated by stuck 405 MHz core clocks.

Sep 24, 2026
Tone: positiveNVIDIA hardware
generation:
70.0 tokens/s
quant:
Q4_K_XL (GGUF)

User reports almost 70 t/s with Qwen3.6-35B-A3B Q4_K_XL on a budget build using three Nvidia Tesla P100 16GB GPUs. The GPUs cost $80 each and are split across three nodes to manage thermals; the motherboard required a patched BIOS to enable Above 4G Decoding. User is still testing and optimizing, and has published a GitHub repository for the build.

Sep 22, 2026
Tone: positiveNVIDIA hardware
generation:
40.0 tokens/s
quant:
Q6_K (GGUF)

User reports Qwen3.8 27B at 40+ t/s on two Tesla P100s, with a roughly 30 t/s average across long contexts and up to 55 t/s at 0 context. Setup is a custom llama.cpp fork with P100 kernel optimizations, Q6_K quant, both cards capped at 175W and communicating over PCIe gen 3. Speeds vary by about ±2 t/s; the user notes fp16 math saves 40% or more on prefill with negligible accuracy loss.

Sep 20, 2026

Get a weekly email of new NVIDIA Tesla P100 16GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Unknown family
2× NVIDIA Tesla P100 16GB
Not reported
Engine not reported
Not reported38.0 tokens/s
Qwen3.8 27B
2× NVIDIA Tesla P100 16GB
UD-Q5_K_XL
llama.cpp
262,14445.0 tokens/s
Qwen3.8 27B Swift-1.5
NVIDIA Tesla P100 16GB
Not reported
Strata
131,07230.0 tokens/s
Qwen3.8 27B
2× NVIDIA Tesla P100 16GB
Q4_0
llama.cpp
Not reported33.3 tokens/s
Qwen3.8 27B
NVIDIA Tesla P100 16GB
Q3_K_M
Engine not reported
Not reported10.0 tokens/s
Qwen3.8 125B (6B active) Flash-Next
NVIDIA Tesla P100 16GB
IQ3_XXS
llama.cpp
80,0005.8 tokens/s

Frequently asked

How fast is the NVIDIA Tesla P100 16GB for local LLMs?

On the NVIDIA Tesla P100 16GB, Qwen3.8 27B at 3-bit, the model most run on it, generates a median of 10.0 tokens per second from one plain run (community reports, as of 22 Nov 2026).