llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5070 Ti · Strata · 131,072 ctx

Tone: mixedNVIDIA hardware
Oct 16, 2026
Throughput
147.0 t/s gen
Quant
IQ2_XS (GGUF)
KV cache
int8
System RAM
64 GB
VRAM reported
16 GB

Use cases

codingtool-useagenticsummarizationlong-context

Summary

User reports Qwen3.8 Flash-Next IQ2_XS at 147 tok/s decode on an RTX 5070 Ti 16 GB with 64 GB DDR5-4800, after running Strata's --calibrate. Setup is Strata v0.1.43 with IQ2_XS GGUF, int8 KV cache, 131k context, --kv-resident 32768, MTP --spec 4, reasoning_budget_tokens 8192, conversation cache 4 GiB, vision encoder on CPU. Calibration raised median decode from 118 to 143-147 tok/s and cut the full 28-task x2 run from 20.6 to 15.4-16.0 min, but the benchmark score dropped from 50 to 44/56 twice, all in soft exact-match cases. Greedy decoding scored 50/56 versus 37 with Qwen's recommended sampling (temp 1.0/top_p 0.95/top_k 20), 43 at temp 0.6 and 40 at temp 1.0 + min_p 0.05. reasoning_effort high scored 42/56 and was twice as slow. Run-to-run noise is about +/-3/56. Real agent workload (Hermes, 54k context) gave about 100-104 tok/s decode with 78-83% expert-cache hit rate before calibration. RAM runs at 4800 MT/s with EXPO off, which the user estimates costs a few to ~20% versus DDR5-6000.