llamaperf

NVIDIA RTX 4070 Ti

NVIDIA · 12GB · 5 reports

Is this your card?

On the NVIDIA RTX 4070 Ti, Qwen3.5 4B at 4-bit, the model most run on it, generates a median of 118.0 tokens per second from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the NVIDIA RTX 4070 Ti, each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

Bars are medians on one scale, 0 to 118.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: llama.cpp 3 (reports)

Run models on your NVIDIA RTX 4070 Ti? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA RTX 4070 Ti

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 12 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Inkling 2 Mini

NVIDIA RTX 4070 Ti · llama.cpp · 65,536 ctx

Tone: positiveNVIDIA hardware
generation:
51-64 tokens/s
quant:
mixB (GGUF)
kv:
q8_0
tool-usevision

User reports Inkling 2 Mini at 51-64 tok/s decode and ~1,500 tok/s prefill at 64K context on a single RTX 4070 Ti 12 GB. Setup is a forked llama.cpp (inkling-fast) with a custom 2-3 bit mixB GGUF (10.3 GiB) and q8_0 KV cache, fitting in 11.6 GB VRAM including the Windows desktop; layers 22-23 run on CPU via -ot. Decode is 61-64 tok/s on short prompts and 51-53 tok/s at 22K; llama-bench pp512/tg128 gives 63.5-63.8 tok/s decode and 2,183-2,191 tok/s prefill. Quality is 90.9% top-1 vs Q8_0, KLD 0.048, PPL ratio 1.034.

Nov 22, 2026

Qwen3.5 4B Fonzie0.2

NVIDIA RTX 4070 Ti · llama.cpp · 65,536 ctx

Tone: mixedNVIDIA hardware
generation:
118.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
q8_0
tool-useagenticmultilingualvision

User reports Fonzie0.2-4B, a LoRA fine-tune of Qwen3.5-4B, decoding about 118 tok/s single-user on an RTX 4070 Ti with llama-server, and about 157 tok/s with MTP 2. Setup is llama.cpp's llama-server with the Q4_K_M GGUF (2.8 GB plus a 0.7 GB vision projector) and a q8_0 KV cache; at 64k context the whole server takes about 5.4 GiB. The model was trained on one DGX Spark with Unsloth on 1,390 conversations in 97 minutes. The user reports accuracy gains in three agent apps but flat accuracy on everyday questions, more invented links (5.9% vs 0.3%), and that GRPO gains in their own harness did not transfer to real apps.

Oct 31, 2026
Tone: mixedNVIDIA hardware
generation:
19.2 tokens/s
quant:
UD-IQ3_XXS (GGUF)

User benchmarks MoMoE, an MIT-licensed fork of Strata, against llama.cpp on Ornith-1.5-122B-A10B UD-IQ3_XXS on an RTX 4070 Ti 12 GB, reporting 19.2 t/s versus 4.6 t/s. MoMoE runs mixture-of-experts models on one GPU plus system RAM, so weights are partly offloaded. The post also lists Qwen3.8-Flash-Next at 65-77 vs 23.1 t/s and Ornith-1.5-35B-A3B at 132-154 vs 77.8 t/s on the same card, both with MTP drafting. User notes llama.cpp ties or wins when a model fits on the GPU, citing LFM2.5-24B at 105 vs 76 t/s and Flint at 103 vs 90 t/s on the PC, and says the fork is early and tested only on their own machines.

Oct 24, 2026
NVIDIA hardware
generation:
69.7 tokens/s
prompt processing (prefill):
6757.3 tokens/s
quant:
Q8_0 (GGUF)

User reports Gemma 4 E4B at 69.66 t/s generation and 6757.3 t/s prompt processing on an NVIDIA RTX 4070 Ti. Setup is llama.cpp with Q8_0 and requested GPU layers 999, a direct full-offload run on a 12GB card. The figures come from a synthetic pp512/tg128 test at zero context depth, so they do not measure generation after a filled context.

Oct 14, 2026

Qwen3.8 27B

NVIDIA RTX 4070 Ti · llama.cpp · 4,096 ctx

NVIDIA hardware
generation:
43.6 tokens/s
prompt processing (prefill):
943.3 tokens/s
quant:
UD-IQ2_XXS (GGUF)
kv:
Q8

User benchmarks Qwen3.8-27B at 43.643 generation tok/s and 943.278 prompt tok/s on an RTX 4070 Ti 12GB. Setup is llama.cpp b10448 with UD-IQ2_XXS GGUF, Q8 KV cache, 4096-token context, all 66 layers on GPU, one slot, 256-token outputs. A second quant UD-Q2_K_XL decoded 38.030 tok/s and 995.095 prompt tok/s with 1,583 MiB more peak VRAM. An IQ2 context ladder gave 41.124 tok/s at 4K, 40.522 at 8K and 39.201 at 16K (12,831 prompt tokens, 11.119 s TTFT). An isolated IQ2 draft-mtp experiment raised throughput 47.284% on prose and 92.651% on Python code, but prose output diverged at token 16, so MTP stays off by default. In a 24-task quality pass, Q2 scored 10/24 and IQ2 9/24 (McNemar p = 1.0).

Sep 28, 2026

Get a weekly email of new NVIDIA RTX 4070 Ti reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3.5 4B Fonzie0.2
NVIDIA RTX 4070 Ti
Q4_K_M
llama.cpp
65,536118.0 tokens/s
Ornith1.5 122B (10B active)
NVIDIA RTX 4070 Ti
UD-IQ3_XXS
MoMoE
Not reported19.2 tokens/s
Gemma 4 8B E4B
NVIDIA RTX 4070 Ti
Q8_0
llama.cpp
Not reported69.7 tokens/s
Qwen3.8 27B
NVIDIA RTX 4070 Ti
UD-IQ2_XXS
llama.cpp
4,09643.6 tokens/s

Frequently asked

How fast is the NVIDIA RTX 4070 Ti for local LLMs?

On the NVIDIA RTX 4070 Ti, Qwen3.5 4B at 4-bit, the model most run on it, generates a median of 118.0 tokens per second from one plain run (community reports, as of 22 Nov 2026). Also reported: Qwen3.8 27B at 2-bit (43.6 t/s, one plain run).

What is the biggest model reported running on the NVIDIA RTX 4070 Ti?

The largest model with a plain generation run on the NVIDIA RTX 4070 Ti is Qwen3.8 27B at 2-bit (43.6 t/s, one plain run), as of 22 Nov 2026. Plain means one device, one request, no speculative decoding, so a run split across several cards does not count.