llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5080 · Strata · 65,536 ctx

Tone: positiveNVIDIA hardware
Oct 21, 2026
Throughput
150.0 t/s gen · 4396.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
KV cache
int8
System RAM
64 GB
VRAM reported
16 GB

Use cases

agentic

Summary

User reports Qwen3.8-Flash-Next at 150 tok/s decode and about 4,400 tok/s prompt on a single RTX 5080 16 GB. Setup is Strata v0.2.1 with IQ3_XXS GGUF, int8 KV cache and 64K context, MTP speculation on, 15 CPU expert workers and an expert cache of 5,310 experts (8.61 GiB VRAM). Decode median 150.0 tok/s over 9 runs (126.7-170.2); fresh prompt runs gave 3,755 tok/s at 4K, 4,396 at 16K and 4,412 at 32K. The card was tuned with LACT (memory +6000, core capped 2850 MHz, 360 W), so stock cards may be slower.