llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 3× NVIDIA RTX 3090 · TabbyAPI · 262,144 ctx

Tone: positiveNVIDIA hardware
Sep 14, 2026
Throughput
120-126 t/s gen
Quant
3.05bpw (EXL3)
MTP (Multi-Token Prediction)
on
System RAM
256 GB
VRAM reported
72 GB

Use cases

codinglong-contexttool-use

Summary

User reports Qwen3.8 Flash-Next at ~120-126 t/s on 3x RTX 3090 (72GB VRAM) with MTP speculative decoding. Setup is TabbyAPI with ExLlamaV3 1.5.0, 3.05bpw exl3 quant, 262,144 context, and MTP draft mode. Without MTP the model runs at ~90 t/s; MTP adds ~35% with ~40% acceptance. First request after boot reads ~58 t/s due to warmup. Needle-in-haystack at ~97k tokens found with ~31s prefill; 4 concurrent requests complete in ~4s each. Tool calling works and reasoning splits into reasoning_content.