llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA V100 32GB · Strata · 131,072 ctx

Tone: positiveNVIDIA hardware
Oct 9, 2026
Throughput
16.7 t/s gen · 61.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
System RAM
128 GB
VRAM reported
32 GB

Use cases

long-contextmathcoding

Summary

User reports Qwen3.8-Flash-Next at 16.7 t/s steady on a Tesla V100-PCIE-32GB, with prompt processing at 61.7 t/s warm. Setup is Strata v0.1.38 compiled for sm_70 with IQ3_XXS, 131,072 context, KV streamed to RAM, MTP draft (q2_0), and 15,553 of 24,576 experts (63%) cached in VRAM (25.2 GiB). Speed stays flat at 16.7-16.9 t/s through a 31k-token session; MTP acceptance is 97.8% on repetitive output and ~65% on free prose. The user also reports a thinking-budget pitfall where hard prompts can exhaust max_tokens on reasoning and return empty content, fixed by sending reasoning_budget_tokens: 4000 per request. A comparison against Qwen3.8-27B on llama.cpp on the same hardware found the 125B stronger on long-tail knowledge and proof depth, while the 27B was 1.7-2x faster in wall clock and better at fine-grained format obedience.