Qwen3.8 125B (6B active) Flash-Next
on 4× NVIDIA RTX 3090 · llama.cpp · 200,000 ctx
Summary
User reports 22 t/s with the Q6_K_XL quant on 4x RTX 3090, compared to 47 t/s for Q4_K_XL.
Setup is llama-server with --split-mode layer, --kv-unified, --flash-attn on, and speculative decoding (ngram-mod). Context is limited to 200k to avoid OOM, with no KV quantization.
User mentions MTP for Qwen 3.8 Flash Next may land upstream, and prefers 27B_Q8 as a daily driver.