llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX Pro 4500 Blackwell 32GB · llama.cpp · 240,000 ctx

Tone: mixedNVIDIA hardware
Sep 22, 2026
Throughput
27.9 t/s gen · 150.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q3_K_XL (GGUF)
KV cache
F16
System RAM
64 GB
VRAM reported
32 GB

Use cases

agenticlong-context

Summary

User reports Qwen3.8 Flash-Next 125B MoE at 27.9 t/s token-weighted decode on an RTX PRO 4500 Blackwell 32GB with 64GB system RAM. Setup is a llama.cpp fork with UD-Q3_K_XL GGUF, 240,000 context, F16 KV cache, 16 of 48 routed-expert layers on GPU and 32 on CPU, disk-backed PLE, and MTP disabled. Real prompt chunks prefill around 150 t/s with a peak of 201. Decode stayed flat from 116K to 148K context (27.7, 28.5, 27.5 t/s across the three bands). The user notes the sparse-QSA CUDA patch is experimental and unreviewed, and that these are warm-cache workload numbers rather than a controlled benchmark.