llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 3090 · Strata

Tone: positiveNVIDIA hardware
Nov 1, 2026
Throughput
170-210 t/s gen · 1500-2000 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
UD-Q5_K_XL (GGUF)
KV cache
F16
System RAM
128 GB

Use cases

agenticcoding

Summary

User reports Qwen3.8-Flash-Next Gyro-M at 170-210 t/s decode and up to 5k t/s prefill on a dual RTX 3090 machine with 128GB DDR4, using Strata with a UD-Q5_K_XL GGUF and F16 KV cache. All experts and KV cache fit in VRAM, with the PLE in system RAM. On a second machine (dual RTX 5070 Ti, 32GB DDR4) the same model ran at 105-115 t/s decode and up to 3k t/s prefill with int8 KV cache, slower to decode than the IQ3_XXS build it replaced but with higher prefill. The user says Gyro-M quality is on par with Q4 and nearly as capable as Q5, with no reasoning loops or file rewrites, and has used it as a subagent for a couple of weeks. The previous IQ3_XXS build ran at 120 t/s decode and 2-2.5k t/s prefill on the 5070 Ti pair, and the Q5 build at 85-95 t/s decode and 1.5-2k t/s prefill on the 3090 pair.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 3090

Bars are medians on one scale, 0 to 111.7 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 3090 →Every Qwen3.8 report →Estimate Qwen3.8 on this card →