llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4070 Super · Strata · 8,192 ctx

Tone: positiveNVIDIA hardware
Oct 14, 2026
Throughput
40.8 t/s gen · 879.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
KV cache
Q4
System RAM
32 GB
VRAM reported
12 GB

Use cases

agenticvisionlong-context

Summary

User reports running Qwen3.8 Flash-Next (125B MoE) on an RTX 4070 Super 12GB with 32GB system RAM, reaching 40.8 t/s decode and 879 t/s prefill at 8K context for the Gyro-S variant. Setup is upstream Strata with source patches, Q4 KV cache with 32K tokens GPU-resident, 256K context window, experts tiered across VRAM, 16 GiB in RAM and the rest streamed from SSD, MTP off. Gyro-M at 8K measured 33.2 t/s decode and 947 t/s prefill; Gyro-S is faster across the sweep, up to 41.9 t/s decode at 128K. Raising prefill chunk size from 7,168 to 8,192 tokens gave an 8-15% prefill gain. Vision and MTP both reduced speed and were left off Gyro-M.