llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4070 Super · Strata · 8,192 ctx

NVIDIA hardware
Oct 14, 2026
Throughput
33.2 t/s gen · 947.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
GGUF (GGUF)
KV cache
Q4
System RAM
32 GB
VRAM reported
12 GB

Summary

User reports running Qwen3.8-Flash-Next (125B MoE) on an RTX 4070 Super 12GB with 32GB system RAM, achieving 33.2 tok/s decode and 947 tok/s prefill at 8K context with the Gyro-M variant. Setup uses Strata with local source patches, GGUF weights, Q4 KV cache with 32K tokens GPU-resident, and MTP disabled. Experts are tiered across VRAM, 16 GiB of RAM, and SSD streaming. Gyro-S variant reaches 40.8 tok/s decode and 879 tok/s prefill at 8K, with vision support. Benchmarks cover 8K to 256K context; decode averages 3 cached runs.