llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 5090 · llama.cpp · 65,536 ctx

NVIDIA hardware
Sep 29, 2026
Throughput
48.0 t/s gen · 1429.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q2_K_XL (GGUF)
System RAM
128 GB
VRAM reported
32 GB

Summary

User reports Qwen3.8-Flash-Next at 48.02 t/s short decode and 1429 t/s prefill on an RTX 5090 32GB with 128GB DDR5 system RAM. Setup is llama.cpp PR #27742 with UD-Q2_K_XL (78.9GB in 3 shards), 64k context, -ngl 999 and -ncmoe 24 keeping the first 24 layers' experts on the CPU, no speculative decoding, single stream. Decode after about 16k tokens of input is 43.28 t/s, time to first token is 11.0 s at about 16k input, and VRAM use is about 30.9GB.