Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5070 Ti · llama.cpp · 262,144 ctx

Sep 28, 2026

Throughput

20.4 t/s gen · 129.3 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

UD-IQ3_XXS (GGUF)

KV cache

Q4_0

Flash Attention

VRAM reported

16 GB

Summary

User reports Qwen3.8-Flash-Next at 20.44 t/s on an RTX 5070 Ti 16GB. Setup is llama.cpp (unsloth build) with UD-IQ3_XXS GGUF and Q4_0 KV cache, 262144 context, 49 layers offloaded, CPU MoE, and ngram-mod speculative decoding. Prompt processing measured at 129.29 t/s over 2409 tokens.

View source ↗