llamaperf

Qwen3.8 125B (6B active) Flash-Next

on NVIDIA RTX 4090 · llama.cpp · 120,000 ctx

NVIDIA hardware
Oct 17, 2026
Throughput
30.0 t/s gen · 864.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-IQ3_XXS (GGUF)
KV cache
q8_0
Flash Attention
on

Use cases

long-context

Summary

User reports 30 t/s generation and 864 t/s prompt processing. Source evidence (verbatim): Source: https://gist.github.com/ryan4yin/48617bbddacc7067f10799770b7cc33f # Qwen3.8-Flash-Next on an RTX 4090 with llama.cpp (mainline), 96GB DDR5 Hardware: RTX 4090 24GB, Intel Core Ultra 7 270K Plus (24C/24T), 96GB DDR5-5600 (2x48GB dual channel). Engine: mainline llama.cpp, image ghcr.io/ggml-org/llama.cpp:full-cuda13. Model: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ3_XXS (about 82GB) plus mmproj-F16. Flags: --ctx-size 120000 --fit on --fit-target 256 -t 20 -tb 24 -b 6144 -ub 6144 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 -np 1. Experts are offloaded to system RAM. No speculative decoding or MTP (llama.cpp does not support the MTP head yet). Measured with this llama.cpp config: decode about 30 t/s near 6K context, 22.2 t/s at 90K context, 19.5 t/s at 110K input; prefill 864 t/s at batch 6144 (762 t/s at batch 2048).