llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5090 · Strata · 262,144 ctx

Tone: mixedNVIDIA hardware
Oct 14, 2026
Throughput
161.3 t/s gen · 5537.8 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_S (GGUF)
KV cache
INT8
System RAM
128 GB
VRAM reported
32 GB

Summary

User reports Qwen3.8-Flash-Next GSQ-RCO IQ3_S at a median 161.3 tok/s generation on an RTX 5090 with 128GB DDR5-6000, measured on three 240,000-token inputs. Setup is Strata 0.1.43.1 with INT8 KV cache, 32,768 resident KV cells, spec 4, draft floor 0.50, 12 pool workers, a 55 GiB resident-expert budget and a 700 MiB VRAM reserve, under a 262,144-token context ceiling. Median prompt processing was 5537.8 tok/s, median first visible answer 43.475 s and median total request time 45.071 s; the generation range was 159.6-203.4 tok/s. The same nine requests at a 131K ceiling gave 173.5, 181.2 and 183.0 tok/s at 4,096, 32,768 and 128,000 input tokens, and the 262K ceiling gave 171.2, 175.0 and 179.2 tok/s. Unsloth UD-IQ4_XS reached a 121.2 tok/s short-task median after tuning, against 105.0 before, with 3/3 exact recall at 100,510 and 128,510 tokens. The user notes the RAM upgrade comparison is not a RAM-only A/B test and that no statistical significance was established.