Qwen3.8 125B (6B active) Flash-Next
on NVIDIA RTX 4090 · llama.cpp · 120,000 ctx
Use cases
long-context
Summary
User reports 30 t/s generation and 864 t/s prompt processing.
Source evidence (verbatim):
Source: https://gist.github.com/ryan4yin/48617bbddacc7067f10799770b7cc33f
# Qwen3.8-Flash-Next on an RTX 4090 with llama.cpp (mainline), 96GB DDR5
Hardware: RTX 4090 24GB, Intel Core Ultra 7 270K Plus (24C/24T), 96GB DDR5-5600 (2x48GB dual channel). Engine: mainline llama.cpp, image ghcr.io/ggml-org/llama.cpp:full-cuda13. Model: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ3_XXS (about 82GB) plus mmproj-F16. Flags: --ctx-size 120000 --fit on --fit-target 256 -t 20 -tb 24 -b 6144 -ub 6144 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 -np 1. Experts are offloaded to system RAM. No speculative decoding or MTP (llama.cpp does not support the MTP head yet). Measured with this llama.cpp config: decode about 30 t/s near 6K context, 22.2 t/s at 90K context, 19.5 t/s at 110K input; prefill 864 t/s at batch 6144 (762 t/s at batch 2048).