llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX A2000 12GB · llama.cpp · 131,072 ctx

NVIDIA hardware
Sep 18, 2026
Throughput
5.5 t/s gen · 118.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
KV cache
q8_0
System RAM
128 GB
VRAM reported
12 GB

Use cases

visionlong-contextsummarization

Summary

User reports Qwen3.8 Flash-Next at 5.5 t/s generation and 118 t/s prompt eval at 131,072 context on an RTX A2000 12GB with a Xeon Silver 4314 and 128 GB DDR4. Setup is llama.cpp master with the unsloth UD-Q4_K_XL GGUF (103.7 GiB), q8_0 KV cache, --n-cpu-moe 99 and -ub 1536, all routed experts in RAM, 11.6 GB VRAM peak. Generation falls from about 15.4 t/s empty to 8.8 t/s at 58k and 5.5 t/s at 118k. A 3000x2000 image request took 46 s. The same box also ran Qwen3.6-35B-A3B at 26.1 t/s and DeepSeek-V4-Flash at 7.5 t/s at 118k deep.