llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA CMP 170HX 8GB · Strata · 512 ctx

NVIDIA hardware
Oct 9, 2026
Throughput
121.0 t/s gen · 630.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
KV cache
int8
MTP (Multi-Token Prediction)
on

Summary

User benchmarks Qwen3.8 Flash-Next at 121 t/s generation and 630 t/s prefill on two CMP 170HX 8GB cards. Setup is Strata with IQ3_XXS GGUF weights, int8 KV cache, MTP speculative decoding with 4 tokens, and a 262144 max context. The table covers prompt lengths from ~0.5K to 140K and concurrency 1 to 4; the reported figures are the c1 ~0.5K row. Total output tok/s at c1 ~0.5K is 89.