Use cases
coding
Summary
User reports Qwen3.8 Flash-Next at 287 tok/s single-stream on one RTX PRO 6000 96GB, with 965 tok/s total at 8 concurrent requests.
Setup is vLLM with FP4 weights, FP8 KV cache, TP1, 262K max context and MTP speculative decoding at depth 4, run at a 450 W power limit.
Prefill is about 14.5K tok/s at 4K context and about 13.5K tok/s at 128K. The throughput is wall-clock and includes prefill; prompts were coding tasks of about 250 tokens in and 1,536 out. The user notes short prose and multilingual chat run a few percent slower.