llamaperf

Qwen3.8 125B (6B active) Flash-Next OrcaRouter

on 2× NVIDIA RTX 3090 · Strata

Tone: positiveNVIDIA hardware
Oct 8, 2026
Throughput
100.0 t/s gen
Quant
IQ4_XS (GGUF)
System RAM
64 GB

Summary

User reports Qwen3.8-Flash-Next-OrcaRouter IQ4_XS at a bit over 100 t/s decode on short context on 2x RTX 3090 24GB with Strata. Setup is Strata on 2x RTX 3090 with IQ4_XS weights, which keeps the useful experts on the GPUs and the rest in 64GB DDR5 system RAM, vision enabled, experimental context extension configured for 512K. A prior vLLM setup did about 60-70 t/s decode. Context extension runs: 200K prompt at ~2,004 t/s prompt processing and ~73 t/s decode; 400K at ~1,633 t/s and ~46 t/s; 900K at ~1,110 t/s and ~44 t/s.