llamaperf

Qwen3.8 125B (6B active) Flash-Next

on RTX 4090 48GB · Strata · 262,144 ctx

Tone: mixed
Oct 9, 2026
Throughput
128-151 t/s gen · 4249-5154 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_S (GGUF)
KV cache
int8
System RAM
128 GB
VRAM reported
48 GB

Summary

User benchmarks Strata 0.1.38 on an RTX 4090 48GB with 128GB DDR5-6000, running Qwen3.8-Flash-Next at IQ3_S with 262K context, int8 KV cache and speculative decoding. Baseline decode is 128-151 tok/s and prefill 4,249-5,154 tok/s at 78K-92K context. Keeping the 28.8 GB n-gram table in RAM gave only +0.65% prefill and +1.2% decode for +28.4 GiB RAM, while conversation parking returned to a 91,836-token conversation in 537 ms instead of 18.8 s for 3.1 GB of RAM. Raising --prefill auto to 32768 gave +9.7% prefill and +13.6% decode at a 78.7K prompt, and --calibrate picked 12 CPU workers for +3.2%. User notes session-to-session spread of 11.5% on unchanged settings, that mmap made the first long prompt 3.7x slower, and that minimizing the engine console on Windows 11 cost 20% decode.