Oct 8, 2026
Summary
User reports Qwen3.8-Flash-Next-OrcaRouter IQ4_XS at a bit over 100 t/s decode on short context on 2x RTX 3090 24GB with Strata.
Setup is Strata on 2x RTX 3090 with IQ4_XS weights, which keeps the useful experts on the GPUs and the rest in 64GB DDR5 system RAM, vision enabled, experimental context extension configured for 512K.
A prior vLLM setup did about 60-70 t/s decode. Context extension runs: 200K prompt at ~2,004 t/s prompt processing and ~73 t/s decode; 400K at ~1,633 t/s and ~46 t/s; 900K at ~1,110 t/s and ~44 t/s.