llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 3090 Ti · Strata · 262,144 ctx

Tone: mixedNVIDIA hardware
Oct 16, 2026
Throughput
148.9 t/s gen · 1886.8 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_S (GGUF)
KV cache
int8
MTP (Multi-Token Prediction)
on
System RAM
59 GB

Use cases

codinglong-context

Summary

User reports Qwen3.8-Flash-Next at 148.86 tok/s narrative decode and 174.67 tok/s code decode on a dual-GPU rig (RTX 3090 Ti + RTX 3090). Setup is Strata with IQ3_S quant, int8 KV cache, 262,144-token context, MTP speculation, --spec 4, --kv-resident 32768, --mmap-experts, vision on, automatic layer split. The 148.86/174.67 figures are the ISO18 profile (PREFILL_HELP), which also gives 1,886.76 tok/s at 10K prefill and 3,641.54 tok/s at 90K. ISO19 (PF_FUSED) is the prefill winner at 1,995.53 tok/s at 10K and 4,149.45 tok/s at 90K, but about 5% slower on narrative decode and 3% slower on code decode; its output-sanity check was still pending. The post traces a Strata 0.1.41 regression (decode about 8% slower, 10K prefill about 15% faster, 90K prefill about 7% slower) to automatic GPU placement, and verifies a fix (STRATA_SPLIT_UNIFORM=1 plus gpu_order: as_given) that restores the 0.1.40.3 baseline range. The fix is opt-in and validated on this specific rig.