Qwen3.8 125B (6B active) Flash-Next
on RTX 4090 48GB · Strata · 262,144 ctx
Summary
User benchmarks Strata 0.1.38 on an RTX 4090 48GB with 128GB DDR5-6000, running Qwen3.8-Flash-Next at IQ3_S with 262K context, int8 KV cache and speculative decoding. Baseline decode is 128-151 tok/s and prefill 4,249-5,154 tok/s at 78K-92K context.
Keeping the 28.8 GB n-gram table in RAM gave only +0.65% prefill and +1.2% decode for +28.4 GiB RAM, while conversation parking returned to a 91,836-token conversation in 537 ms instead of 18.8 s for 3.1 GB of RAM. Raising --prefill auto to 32768 gave +9.7% prefill and +13.6% decode at a 78.7K prompt, and --calibrate picked 12 CPU workers for +3.2%.
User notes session-to-session spread of 11.5% on unchanged settings, that mmap made the first long prompt 3.7x slower, and that minimizing the engine console on Windows 11 cost 20% decode.