Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX A2000 12GB · llama.cpp · 131,072 ctx
Sep 18, 2026
Use cases
visionlong-contextsummarization
Summary
User reports Qwen3.8 Flash-Next at 5.5 t/s generation and 118 t/s prompt eval at 131,072 context on an RTX A2000 12GB with a Xeon Silver 4314 and 128 GB DDR4.
Setup is llama.cpp master with the unsloth UD-Q4_K_XL GGUF (103.7 GiB), q8_0 KV cache, --n-cpu-moe 99 and -ub 1536, all routed experts in RAM, 11.6 GB VRAM peak.
Generation falls from about 15.4 t/s empty to 8.8 t/s at 58k and 5.5 t/s at 118k. A 3000x2000 image request took 46 s. The same box also ran Qwen3.6-35B-A3B at 26.1 t/s and DeepSeek-V4-Flash at 7.5 t/s at 118k deep.