llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX Pro 4500 Blackwell 32GB · Strata · 200,000 ctx

Tone: positiveNVIDIA hardware
Oct 19, 2026
Throughput
60-78 t/s gen
Quant
FP4-Local QAD (GGUF)
KV cache
INT8
System RAM
64 GB
VRAM reported
32 GB

Use cases

tool-use

Summary

User reports Qwen3.8-Flash-Next FP4-Local at roughly 60-78 tok/s around 40-55k real context on a single RTX PRO 4500 Blackwell 32GB with 64GB DDR5. Setup is Strata 0.1.45 with NVIDIA FP4-Local QAD weights, INT8 KV cache at 200k context, a ~38GB RAM expert tier with adaptive VRAM expert cache, SSD-backed PLE and MTP speculative decoding (spec=6). Cold 43k prompt prefill was about 1,890 tok/s with up to ~90% GPU expert-cache hit. BFCL-v4 quick test scored 85% overall and 92% on the 50 single-turn tool-selection tasks.