llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5070 · llama.cpp · 131,072 ctx

Tone: positiveNVIDIA hardware
Sep 16, 2026
Throughput
15.0 t/s gen · 104.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
System RAM
64 GB
VRAM reported
12 GB

Summary

User reports Qwen3.8 Flash-Next at 15 t/s on an RTX 5070 12GB, with prompt processing at 100-120 t/s. Setup is llama.cpp with a 3 bpw IQ3_XXS GGUF (76GB total, 47GB sharded into VRAM+RAM, rest on SSD) at up to 128K context. Generation held 13.8 t/s at 1K, 14.6 t/s at 8K, 14.0 t/s at 20K, and 11.3 t/s at 90-100K context. User rates output quality as matching Unsloth Q6-Q8.