Local model performance on your hardware

generation:

9-11 tokens/s

prompt processing (prefill):

400-550 tokens/s

quant:

UD_Q4_K_XL (GGUF)

kv:

q8_0

User reports Qwen 3.8 27B at ~9-11 t/s decode and 400-550 t/s prefill on an RTX 5070 Ti Laptop 12GB. Setup is llama.cpp with UD_Q4_K_XL quant, q8_0 KV cache, 98304 context, draft-mtp speculative decoding, and FFN weights offloaded to CPU. The user notes reasoning verbosity blows past 100K tokens in planning, and with Magic Context the session scaled past 3.7M total processed tokens.

generation:

59.0 tokens/s

prompt processing (prefill):

155.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

User compares Qwen3.8 27B dense at Q2 and Q3 against Qwen3.6 35B-A3B MoE on 12 GB of VRAM. The MoE was fastest and passed a sanity test, while dense Q3 ran at 7.5-9.1 t/s. The user prefers the MoE for local use.

generation:

5.0 tokens/s

quant:

Q4_K_XL (GGUF)

kv:

Q8

User reports Qwen 3.8 27B UD Q4_K_XL on an RTX 5070 Ti Mobile 12GB with llama.cpp, at roughly 1.5-5 t/s in a context-prioritized configuration and roughly 9-11.5 t/s in a speed-prioritized one. Setup uses a q8_0 KV cache, with context at 262144 in the first configuration and 98304 in the second. User praises the model's capability and instruction following.