Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 4070 · llama.cpp · 98,000 ctx
Oct 3, 2026
Summary
User reports Qwen3.8 Flash-Next at 15.7 t/s on an RTX 4070 12GB and V100 16GB with 32GB of system RAM.
Setup is llama.cpp with IQ3_XXS weights, 98k context, flash attention, jinja, and MTP with 2 draft tokens, using layer split rather than tensor parallelism.
The user asks whether the model should require 45.8 GB in memory as shown for an AD-3.84bpw-IQ4_XS-M64 build, and notes their own RAM usage stayed around 6 GB.