llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5070 Ti · Strata · 262,144 ctx

Tone: mixedNVIDIA hardware
Oct 11, 2026
Throughput
81.9 t/s gen
Quant
IQ3_S (GGUF)
KV cache
int8
System RAM
64 GB
VRAM reported
16 GB

Use cases

agentictool-uselong-contextsummarizationcoding

Summary

User reports Qwen3.8-Flash-Next IQ3_S at 81.9 tok/s in calibration and 85.4 tok/s in chat on an RTX 5070 Ti 16GB with 64GB RAM. Setup is Strata with IQ3_S GGUF, int8 KV cache, 256K context, KV streaming with 32768 resident cells, and MTP speculation. The model is 125B-A6B with 46.84 GiB of experts loaded and an 84GB download. Two concurrent 256K slots exceeded a 58 GiB container limit during startup with and without batch MTP, and earlier unrestricted runs caused global OOMs and a host freeze. The user settled on one active request at 256K with KV streaming.