llamaperf

Qwen3.8 125B (6B active) Swift-1.5

on NVIDIA RTX 3090 · Strata · 262,144 ctx

Tone: positiveNVIDIA hardware
Oct 21, 2026
Throughput
110-120 t/s gen · 2000-2500 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_S (GGUF)
System RAM
128 GB
VRAM reported
24 GB

Summary

User reports Qwen3.8 Flash-Next Swift-1.5 at 110-120 tok/s decode and 2000-2500 tok/s prefill on an RTX 3090 with Strata. Setup is Strata with IQ3_S at 262144 context on the 24GB card in a 128GB DDR4 system. The same model on an RTX 4060 office PC with IQ2_XS and Q4 KV cache gives around 37-38 tok/s decode and 250 tok/s prefill. On an RTX 5060 with 32GB RAM, Strata prefill was around 100 tok/s. The user says hand-tuned llama.cpp only reached 30-40 tok/s decode, while Strata with the tiel variant gives at least 60 tok/s.