Qwen3.8 125B (6B active) Flash-Next
on Unknown GPU · Strata
Use cases
codingagentic
Summary
User reports solo decode averaging 61 tok/s over 1,677 requests since 12 Nov on a multi-GPU setup running Qwen3.8 Flash-Next in Strata.
Setup uses parallel 2 (two concurrent trains), a tuned layer_split instead of auto, STRATA_NO_LARGEPAGES, and --prompt-cache-every 0 for KV cache reuse; each extra train slot costs about 0.4–0.6 GiB VRAM per card.
User says 78% of routed experts hit VRAM, 3% go over PCIe and 19% run on the CPU, and that rebalancing the layer split should roughly halve CPU expert misses with decode perhaps +10–15%. No GPU model is named.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Qwen3.8 125B · 6B active at 4-bit by hardware
- 129.2 t/s · one run
- 105.5 t/s · 6 runs
- 57.2 t/s · 2 runs
- 55.0 t/s · one run
- 36.7 t/s · one run
- NVIDIA DGX Spark128 GB29.8 t/s · 2 runs
Bars are medians on one scale, 0 to 232.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.