llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 4× NVIDIA RTX 3090 · Strata · 256,000 ctx

NVIDIA hardware
Nov 12, 2026
Throughput
139-144 t/s gen
Quant
UD-Q4_K_XL (GGUF)
KV cache
int8
System RAM
128 GB
VRAM reported
24 GB

Use cases

long-context

Summary

User benchmarks Qwen3.8-Flash-Next (UD-Q4_K_XL, Q8_0 n-gram table) on 4x RTX 3090 24 GB via Strata, reaching 139-144 tok/s on short prompts and 85-96 tok/s after 50K context with all four cards. Setup is Strata 0.2.13 with layer split, DFlash2 drafter, 256K context, int8 KV, one request at a time, on a Threadripper 2920X with 128 GB RAM; experts that miss VRAM are computed on the CPU. Three cards give 112-137 tok/s and two cards 90-95 tok/s; a fourth card mainly speeds prompt reading, which is twice as fast as on two or three cards. Q5_K_XL decodes 28% slower than Q4 at 0.55% lower perplexity. A cold 190K prompt takes 91 s, after which a short turn at 200K decodes at 78 tok/s. Ranges are two runs each.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 3090

Bars are medians on one scale, 0 to 111.7 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 3090 →Every Qwen3.8 report →Estimate Qwen3.8 on this card →