llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 2080 Ti 22GB (modded) · Strata · 128,000 ctx

Tone: positiveNVIDIA hardware
Nov 20, 2026
Throughput
60.1 t/s gen · 28.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
IQ3_S (GGUF)
System RAM
64 GB
VRAM reported
44 GB

Summary

User reports Qwen3.8-Flash-Next at 60.1 tok/s decode and 28 tok/s prefill on two modded RTX 2080 Ti 22GB cards, with a peak of 70.9 tok/s at 61K of 128K context. Setup is Strata with IQ3_S weights, custom tensor parallelism over an NVLink bridge on M.2 OCuLink PCIe Gen3 x4, on a Minisforum AR900i with a Core i9-13900HX and 64GB DDR5. The 60.1 tok/s figure comes from 41,288 tokens in 695 s at 41K context; the internal RTX 5070 was idle during these runs.