NVIDIA RTX 5070 Ti reports

generation:

18-20 tokens/s

quant:

IQ4_XS (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at ~18-20 t/s generation speed on 16 GB VRAM and 64 GB system RAM. Setup uses FFN offloading to keep attention on the GPU, a q8_0 KV cache at 65k context, MTP speculative decoding, and thread pinning.

generation:

36.0 tokens/s

prompt processing (prefill):

560.0 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports llama.cpp RPC across a 5070 Ti and a 1080 Ti over gigabit ethernet, running Qwen 3.8 27B at UD-Q4_K_XL with a Q8 KV cache and 96k context. Best generation was 36 t/s with MTP enabled and GPU order RPC0,CUDA0. A prefill-focused config gave 560 pp and 19 tg. UD-Q5_K_XL with a full KV cache gave 380 pp and 15 tg. User notes MTP tanks prefill speeds and uses VRAM, and that enabling KV quant for the draft model reduces context. User recommends jumbo frames and careful GPU ordering.

generation:

22.3 tokens/s

prompt processing (prefill):

1003.9 tokens/s

quant:

Q4_K_XL (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 22.30 t/s generation and 1003.85 t/s prefill on an RTX 5070 Ti Laptop 12GB plus an RTX 5060 Ti 16GB eGPU. Setup is llama.cpp with layer-split across the two cards. Q4_K_XL measures 1003.85/22.30 t/s pp512/tg128, Q5_K_XL 923.21/19.11, and Q6_K 901.18/18.23. MTP speculative decoding raises generation to 32-38 t/s. The context ceiling is 49,152 for Q6_K and 73,728 for Q5_K_XL. The user also notes tensor-split syntax differences and eGPU power interruption.

generation:

22.0 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports the AtomicChat build of Qwen3.8-Flash-Next running a full 256K context on 96 GB RAM and 40 GB VRAM, at 22 t/s. The build splits the n-gram table to SSD. The user previously ran unsloth quants, which crashed at around 60K context and were slower at 12-14 t/s. The AtomicChat build handles a 217K token conversation without crashing. Prefill takes about 18 minutes for 220K context.

generation:

75.0 tokens/s

prompt processing (prefill):

1700.0 tokens/s

quant:

Q3_K_XL (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B-UD-Q3_K_XL.gguf at up to 75 t/s tg and up to 1700 t/s pp on an RTX 5070 Ti 16GB, fully offloaded, with vision on CPU. Setup is beellama.cpp with kvarn performance enhancements, MTP n-max=2, KV=kvarn3/kvarn3, MTP draft KV=kvarn2/kvarn2, 96256 context and 1024 tail tokens. The user notes Q3_K_XL scores almost 100% on a needle test with kvarn3, while Q3_XXS does poorly at 84% with F16 KV and ~80% with Q3 KV, and that kvarn is better with KLD. The user advises not fearing Q3 quants or KV quantisation, and concludes that for 16GB VRAM Q3_K_XL with kvarn3 works well.

generation:

60.0 tokens/s

prompt processing (prefill):

1500.0 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports 60 t/s decode and 1500 t/s prefill with Qwen3.8-27B-IQ4-XS-MTP across an RTX 5070 Ti and an RTX 5060 Ti. Setup uses tensor parallelism. User is considering adding a 3090 FE for more VRAM but is concerned about mixing Ampere with Blackwell.

generation:

115.0 tokens/s

quant:

Q4_K_XL (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B on dual RTX 5070 Ti 16GB cards. Setup is llama.cpp with the Q4_K_XL quant, MTP3 speculative decoding, and 220k context. The user also tests vLLM and NInfer, and praises llama.cpp for its simplicity and performance.

generation:

26.0 tokens/s

quant:

IQ3_XSS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 26t/s on an RTX 5070 Ti 16GB with 64GB of DDR5 6000MHz system RAM. Setup is llama.cpp with the IQ3_XSS quant, using mmap to fit the model into 16G of VRAM plus 64G of RAM, on a 9800X3D CPU. The user says this is faster than a non-MoE 30B model, which only reaches 10 t/s.

generation:

50-60 tokens/s

quant:

IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User switched from Qwen3.6 35b-a3b IQ4_XS to Qwen3.6 27b IQ3_M. The 35b-a3b ran at 50-60 t/s with slow prompt processing, while the 27b ran at ~40 t/s with consistent performance. User reports the 27b found a bug the 35b could not, and notes the dense model handles compression better than the MoE.

generation:

19.2 tokens/s

prompt processing (prefill):

186.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports running Qwen3.6-27B Q4_K_M on an RTX 5070 Ti 16GB and an RTX 2060 6GB with llama-server, reaching pp 186.76 t/s and tg 19.21 t/s at 71k actual context. Setup is llama-server with the Q4_K_M quant across the two cards. llama-bench results with CUDA show tg speeds around 16-25 t/s depending on configuration.