Qwen3.8 125B (6B active) Flash-Next
on NVIDIA V100 32GB · Strata · 131,072 ctx
Use cases
long-contextmathcoding
Summary
User reports Qwen3.8-Flash-Next at 16.7 t/s steady on a Tesla V100-PCIE-32GB, with prompt processing at 61.7 t/s warm.
Setup is Strata v0.1.38 compiled for sm_70 with IQ3_XXS, 131,072 context, KV streamed to RAM, MTP draft (q2_0), and 15,553 of 24,576 experts (63%) cached in VRAM (25.2 GiB).
Speed stays flat at 16.7-16.9 t/s through a 31k-token session; MTP acceptance is 97.8% on repetitive output and ~65% on free prose. The user also reports a thinking-budget pitfall where hard prompts can exhaust max_tokens on reasoning and return empty content, fixed by sending reasoning_budget_tokens: 4000 per request. A comparison against Qwen3.8-27B on llama.cpp on the same hardware found the 125B stronger on long-tail knowledge and proof depth, while the 27B was 1.7-2x faster in wall clock and better at fine-grained format obedience.