Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX Pro 6000 Blackwell · ik_llama.cpp · 200,000 ctx
Summary
User reports prefill around 2000 t/s and decode around 40 t/s for a single request.
With 2-4 parallel requests, prefill drops to 500-1000 t/s and decode to 10-20 t/s.
User asks for a better setup for 2-4 slots with q4 quants or better and q8 KV cache or better, and mentions trying vLLM recipes before.