llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4090 · Strata · 145,000 ctx

Tone: mixedNVIDIA hardware
Oct 25, 2026
Throughput
95-105 t/s gen · 2076.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IQ3_S (GGUF)
KV cache
int8
System RAM
48 GB
VRAM reported
24 GB

Use cases

coding

Summary

User reports Qwen3.8-Flash-Next at 2076 tok/s prompt processing on a 145k-token prompt and 95-105 tok/s decode depending on task, with peaks of 145 tok/s, on a single RTX 4090 24 GB. Setup is Strata 0.1.43 with an IQ3_S pack, int8 KV cache, 204800 context, resident experts and MTP speculation, on a Ryzen 9 3900X with 48 GiB DDR4-3200 under Windows 10. A grid search over flags raised decode from 97 to 142 tok/s; the best arm was --kv-resident 20480 plus --adapt-async 1 and STRATA_ADAPT_LAG=2, giving 1526 and 2309 tok/s prompt processing at 10k and 20k. The user notes Windows does not pin memory with CUDA, so every host-to-GPU expert copy is CPU-driven.