Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 5060 Ti 16GB · Unsloth Studio · 65,536 ctx
Summary
User reports Qwen3.8-Flash-Next at 9.5-10 t/s decode and 12-38 t/s prefill on dual RTX 5060 Ti 16GB cards with 96GB DDR4 system RAM.
Setup is Unsloth Studio (llama-server backend) with UD-Q3_K_XL quant, q4_0 KV cache, 65k context, experts offloaded via --cpu-moe -ngl 99 -b 1024 -ub 512 -fa on, and MTP speculative decoding with an ~87% draft acceptance rate.
User asks whether ditching MTP, bypassing mmap with --load-mode none, or building a custom llama.cpp with the LRU expert cache would help on DDR4, or whether the setup is bandwidth-capped.