llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× 2x Quadro RTX 5000 · Strata · 131,072 ctx

Tone: positive
Oct 16, 2026
Throughput
44.5 t/s gen
Quant
IQ3_S (GGUF)
KV cache
int8
System RAM
121 GB

Use cases

agenticcodinglong-context

Summary

User reports Qwen3.8-Flash-Next IQ3_S at 44.5 token-weighted t/s on 2x Quadro RTX 5000 with 121 GiB RAM. Setup is Strata with IQ3_S GGUF (79 GB), KV int8, 131072 context, spec 4 MTP speculative decoding, resident-experts mode. User ranks this the best all-round agent of the quants tested: 9 of 10 tasks finished RC=0 with verified deliverables, built a full board game overnight. Per-request decode averaged 49.3 t/s (median 50.4), prefill 250.5 t/s average. During real agentic runs with reasoning on, decode was 33-40 t/s. IQ3_S only fit in full residency on the 121 GiB machine; it did not fit on the 62 GB or 30 GB machines.