Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX Pro 4500 Blackwell 32GB · llama.cpp · 240,000 ctx
Sep 22, 2026
Use cases
agenticlong-context
Summary
User reports Qwen3.8 Flash-Next 125B MoE at 27.9 t/s token-weighted decode on an RTX PRO 4500 Blackwell 32GB with 64GB system RAM.
Setup is a llama.cpp fork with UD-Q3_K_XL GGUF, 240,000 context, F16 KV cache, 16 of 48 routed-expert layers on GPU and 32 on CPU, disk-backed PLE, and MTP disabled. Real prompt chunks prefill around 150 t/s with a peak of 201.
Decode stayed flat from 116K to 148K context (27.7, 28.5, 27.5 t/s across the three bands). The user notes the sparse-QSA CUDA patch is experimental and unreviewed, and that these are warm-cache workload numbers rather than a controlled benchmark.