llamaperf

NVIDIA RTX 4070 Ti reports

5 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX 4070 Ti speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Inkling 2 Mini

NVIDIA RTX 4070 Ti · llama.cpp · 65,536 ctx

Tone: positiveNVIDIA hardware
generation:
51-64 tokens/s
quant:
mixB (GGUF)
kv:
q8_0
tool-usevision

User reports Inkling 2 Mini at 51-64 tok/s decode and ~1,500 tok/s prefill at 64K context on a single RTX 4070 Ti 12 GB. Setup is a forked llama.cpp (inkling-fast) with a custom 2-3 bit mixB GGUF (10.3 GiB) and q8_0 KV cache, fitting in 11.6 GB VRAM including the Windows desktop; layers 22-23 run on CPU via -ot. Decode is 61-64 tok/s on short prompts and 51-53 tok/s at 22K; llama-bench pp512/tg128 gives 63.5-63.8 tok/s decode and 2,183-2,191 tok/s prefill. Quality is 90.9% top-1 vs Q8_0, KLD 0.048, PPL ratio 1.034.

Nov 22, 2026

Qwen3.5 4B Fonzie0.2

NVIDIA RTX 4070 Ti · llama.cpp · 65,536 ctx

Tone: mixedNVIDIA hardware
generation:
118.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
q8_0
tool-useagenticmultilingualvision

User reports Fonzie0.2-4B, a LoRA fine-tune of Qwen3.5-4B, decoding about 118 tok/s single-user on an RTX 4070 Ti with llama-server, and about 157 tok/s with MTP 2. Setup is llama.cpp's llama-server with the Q4_K_M GGUF (2.8 GB plus a 0.7 GB vision projector) and a q8_0 KV cache; at 64k context the whole server takes about 5.4 GiB. The model was trained on one DGX Spark with Unsloth on 1,390 conversations in 97 minutes. The user reports accuracy gains in three agent apps but flat accuracy on everyday questions, more invented links (5.9% vs 0.3%), and that GRPO gains in their own harness did not transfer to real apps.

Oct 31, 2026
Tone: mixedNVIDIA hardware
generation:
19.2 tokens/s
quant:
UD-IQ3_XXS (GGUF)

User benchmarks MoMoE, an MIT-licensed fork of Strata, against llama.cpp on Ornith-1.5-122B-A10B UD-IQ3_XXS on an RTX 4070 Ti 12 GB, reporting 19.2 t/s versus 4.6 t/s. MoMoE runs mixture-of-experts models on one GPU plus system RAM, so weights are partly offloaded. The post also lists Qwen3.8-Flash-Next at 65-77 vs 23.1 t/s and Ornith-1.5-35B-A3B at 132-154 vs 77.8 t/s on the same card, both with MTP drafting. User notes llama.cpp ties or wins when a model fits on the GPU, citing LFM2.5-24B at 105 vs 76 t/s and Flint at 103 vs 90 t/s on the PC, and says the fork is early and tested only on their own machines.

Oct 24, 2026
NVIDIA hardware
generation:
69.7 tokens/s
prompt processing (prefill):
6757.3 tokens/s
quant:
Q8_0 (GGUF)

User reports Gemma 4 E4B at 69.66 t/s generation and 6757.3 t/s prompt processing on an NVIDIA RTX 4070 Ti. Setup is llama.cpp with Q8_0 and requested GPU layers 999, a direct full-offload run on a 12GB card. The figures come from a synthetic pp512/tg128 test at zero context depth, so they do not measure generation after a filled context.

Oct 14, 2026

Qwen3.8 27B

NVIDIA RTX 4070 Ti · llama.cpp · 4,096 ctx

NVIDIA hardware
generation:
43.6 tokens/s
prompt processing (prefill):
943.3 tokens/s
quant:
UD-IQ2_XXS (GGUF)
kv:
Q8

User benchmarks Qwen3.8-27B at 43.643 generation tok/s and 943.278 prompt tok/s on an RTX 4070 Ti 12GB. Setup is llama.cpp b10448 with UD-IQ2_XXS GGUF, Q8 KV cache, 4096-token context, all 66 layers on GPU, one slot, 256-token outputs. A second quant UD-Q2_K_XL decoded 38.030 tok/s and 995.095 prompt tok/s with 1,583 MiB more peak VRAM. An IQ2 context ladder gave 41.124 tok/s at 4K, 40.522 at 8K and 39.201 at 16K (12,831 prompt tokens, 11.119 s TTFT). An isolated IQ2 draft-mtp experiment raised throughput 47.284% on prose and 92.651% on Python code, but prose output diverged at token 16, so MTP stays off by default. In a 24-task quality pass, Q2 scored 10/24 and IQ2 9/24 (McNemar p = 1.0).

Sep 28, 2026