Local model performance on your hardware

reported speed:

89.7 tokens/s prompt processing

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports prefill speed on an RX 6600 XT 8GB that does not fall steadily with context, using llama.cpp Vulkan with every layer offloaded: 935 t/s at about 1K tokens down to 318 t/s at 8K, then 472 t/s at 16K, and 89.7 t/s at 130K against 76.7 t/s at 65K. The model is not named. User says the spread across runs was under 2%.

reported speed:

30.0 tokens/s generation

quant:

UD_Q4_K_XL (GGUF)

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User asks what performance P100 owners get, having ordered one for $80, and reports their current baseline on an RX 6600 XT with 32GB DDR4 3600. Current setup runs unsloth Qwen3.6 35B-A3B UD_Q4_K_XL in llama.cpp with MTP and --cpu-moe at 64k full-precision context, giving 30 t/s generation and 48-50 t/s decode, with prefill up to 800 t/s at 0 context and 700 t/s at 10k. User hopes the P100 can match the decode numbers and plans to tune -b and -ub for its higher core count; the card will go into a dedicated inference machine.