Qwen3.8 125B (6B active) Flash-Next
2× NVIDIA CMP 170HX 8GB · Strata · 512 ctx
- generation:
- 121.0 tokens/s
- prompt processing (prefill):
- 630.0 tokens/s
- quant:
- IQ3_XXS (GGUF)
- kv:
- int8
- mtp (multi-token prediction):
- on
User benchmarks Qwen3.8 Flash-Next at 121 t/s generation and 630 t/s prefill on two CMP 170HX 8GB cards. Setup is Strata with IQ3_XXS GGUF weights, int8 KV cache, MTP speculative decoding with 4 tokens, and a 262144 max context. The table covers prompt lengths from ~0.5K to 140K and concurrency 1 to 4; the reported figures are the c1 ~0.5K row. Total output tok/s at c1 ~0.5K is 89.