llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 3070 · Strata · 1,000 ctx

Tone: positiveNVIDIA hardware
Oct 24, 2026
Throughput
40.0 t/s gen · 1000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_S (GGUF)
KV cache
int8
MTP (Multi-Token Prediction)
on
System RAM
64 GB
VRAM reported
8 GB

Summary

User reports Qwen3.8-Flash-Next at about 40 t/s decode and 1000 t/s prefill on an RTX 3070. Setup is Strata with an IQ3_S GGUF, int8 KV cache, 262144 max context, spec set to 4, and a q8 n-gram table instead of the bundled iq4_nl one. The model is 125B-A6B with experts partly offloaded, on a Ryzen 5 5600 with 64GB of RAM. A vanilla config at the same context gave about 500 t/s prefill and 2-3 t/s less decode. The GPU unlock gave maybe 50-100 t/s extra prefill. The model runs from a SATA SSD, which the user thinks may hamper performance.