llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× NVIDIA RTX 5060 Ti 16GB · Unsloth Studio · 65,536 ctx

Tone: mixedNVIDIA hardware
Sep 26, 2026
Throughput
9.5-10 t/s gen · 12-38 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q3_K_XL (GGUF)
KV cache
q4_0
System RAM
96 GB
VRAM reported
32 GB

Summary

User reports Qwen3.8-Flash-Next at 9.5-10 t/s decode and 12-38 t/s prefill on dual RTX 5060 Ti 16GB cards with 96GB DDR4 system RAM. Setup is Unsloth Studio (llama-server backend) with UD-Q3_K_XL quant, q4_0 KV cache, 65k context, experts offloaded via --cpu-moe -ngl 99 -b 1024 -ub 512 -fa on, and MTP speculative decoding with an ~87% draft acceptance rate. User asks whether ditching MTP, bypassing mmap with --load-mode none, or building a custom llama.cpp with the LRU expert cache would help on DDR4, or whether the setup is bandwidth-capped.