llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 3090 · Strata · 65,536 ctx

Tone: mixedNVIDIA hardware
Nov 3, 2026
Throughput
120-151 t/s gen
Quant
IQ2_XS (GGUF)
System RAM
64 GB
VRAM reported
24 GB

Use cases

tool-useagentic

Summary

User reports Qwen3.8 Flash-Next served by Strata at IQ2_XS with 64k context on a single RTX 3090, running at 120-151 tok/s with the GPU at 53°C and no throttling. Setup is an Unraid box with an Intel i7-9700K and 64 GB DDR4, with Open WebUI calling Strata directly and via Hermes-Agent, plus Home Assistant's MCP server for tools; the fixed prompt was trimmed from about 9,500 to 4,560 tokens. The user asks why new chats are slow while follow-ups are fast: fresh-chat tasks take 8-26 s against 1-2 s for repeats, and Strata's log shows 4,552 of 4,557 prompt tokens reused from cache even in a new chat. Suspected causes are background jobs after the first reply, a redundant first tool call, and per-chat MCP tool setup, none confirmed.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 3090

Bars are medians on one scale, 0 to 111.7 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 3090 →Every Qwen3.8 report →Estimate Qwen3.8 on this card →