llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 3090 · Strata · 262,144 ctx

Tone: mixedNVIDIA hardware
Oct 12, 2026
Throughput
59-62 t/s gen · 1325.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_XXS (GGUF)
KV cache
int8
System RAM
32 GB
VRAM reported
24 GB

Use cases

long-context

Summary

User reports Qwen3.8-Flash-Next 125B-A6B at IQ3_XXS running at the full 262,144-token context on a single RTX 3090 with 32 GB of DDR5, using the Strata engine with int8 KV cache and MTP speculation. On a 121,566-token prompt the run reached 1325 tok/s prefill and 59-62 tok/s decode; a 249K-token prompt gave 1008 tok/s prefill and 41.8 tok/s decode with 90.5% expert-cache hits, and short-context decode climbs to 63-100 tok/s. Setup uses --resident-cpu-experts with a 21 GiB resident budget, 12.2 GiB of experts in VRAM, 21 GiB locked in RAM and 6.6 GiB left on disk, covering 83% of routings without disk reads. The process is pinned to P-cores with taskset -c 0-15 and --pool-affinity p-cores; --prefill auto:16384 raised prefill from 1014 to 1325 tok/s on the 121K prompt. --aux-cpus gave no gain on this chip (60.1 vs 59.6 tok/s decode). The user asks whether these numbers are good and what others get on comparable hardware, and notes the KV cache is about 3.1 GiB at 262K because only 12 of 48 layers keep growing KV.