llamaperf

Qwen3.8 125B (6B active) Flash-Next

on Unknown GPU · Strata

Tone: positive
Nov 22, 2026
Throughput
61.0 t/s gen

Use cases

codingagentic

Summary

User reports solo decode averaging 61 tok/s over 1,677 requests since 12 Nov on a multi-GPU setup running Qwen3.8 Flash-Next in Strata. Setup uses parallel 2 (two concurrent trains), a tuned layer_split instead of auto, STRATA_NO_LARGEPAGES, and --prompt-cache-every 0 for KV cache reuse; each extra train slot costs about 0.4–0.6 GiB VRAM per card. User says 78% of routed experts hit VRAM, 3% go over PCIe and 19% run on the CPU, and that rebalancing the layer split should roughly halve CPU expert misses with decode perhaps +10–15%. No GPU model is named.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3.8 125B · 6B active at 4-bit by hardware

Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Every Qwen3.8 report →