RTX 4070 Ti Super

reported speed:

54.1 tokens/s generation · 1522.4 tokens/s prompt processing

quant:

IQ2_XXS (gguf)

kv:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a full 256k context offloaded to VRAM with no CPU spillover, using n-gram speculative decoding. Generation speed degrades with context depth: 43.69 t/s at 32k, 36.59 t/s at 64k, 27.55 t/s at 128k, 22.15 t/s at 192k, and 18.48 t/s at 256k. Prompt processing also degrades similarly.

reported speed:

5.7 tokens/s generation

quant:

Q4_K_M (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a 4070 Super run that was slow because only 38 of 66 layers were on GPU, with the rest running from system RAM. The same model achieved 81.5 tok/s on a 5090. The user introduces a tool called Picchio for diagnosing layer placement and speeds.

reported speed:

7.6 tokens/s generation

quant:

Q4_K_M (gguf)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Qwen 3.8 27B Q4_K_M running on an RTX 4070 Ti Super 12GB and a MacBook M5 Air over RPC with llama.cpp, completing a coding game clone task in 117 minutes. Setup uses llama.cpp over RPC between the two machines. LM Studio is also mentioned. The 117 minutes is the total time for the task, not a throughput figure.

reported speed:

48.5 tokens/s generation

quant:

IQ4_XS (gguf)

kv:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a custom hybrid quantization that fits MTP and long context into 16 GB of VRAM, with VRAM usage of ~15.93 GB. Setup uses an asymmetric KV cache mix of kvarn5 (K) and kvarn4 (V), with tail precision of 1024 tokens and speculative decoding with 2 draft tokens.

reported speed:

110.2 tokens/s generation

quant:

IQ4_XS-4.19bpw (gguf)

kv:

Q8

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks llama.cpp at 89.76 t/s against ik_llama.cpp at 110.24 t/s on Qwen3.6-35B-A3B with MTP. Setup is the IQ4_XS quant on a Ryzen 7 9700X running CachyOS, with the GPU used as a secondary card and the iGPU for display. The ik_llama.cpp run is a 23% speed increase.