Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA A100 80GB · llama.cpp · 1,835,008 ctx
Use cases
codingagenticlong-context
Summary
User reports Qwen3.8 Flash-Next at 50-70 t/s single-user on dual A100 80GB, dropping to 10-15 t/s with 2-3 concurrent sessions.
Setup is llama.cpp with UD-Q4_K_XL and Q8_0 KV cache, 7 parallel slots at 262k context each (1835008 total), Q8_0 MTP head with draft max 2, mmproj enabled, flash attention on, layer split, 32GB cache RAM.
User is frustrated with multi-user scaling and asks whether to switch to vLLM or run Qwen3.6-35B-A3B per card; use case is long-session agentic coding.