Qwen3.8 125B (6B active) Flash-Next
on 2× NVIDIA RTX 3090 · llama.cpp · 131,072 ctx
Summary
User reports Qwen3.8 Flash Next at 130-200 t/s prefill and 14-22 t/s generation (about 15 t/s on average) on dual RTX 3090s.
Setup is llama.cpp with Q4_K_XL GGUF and q8_0 KV cache at 131072 context, 48GB VRAM across two cards, 128GB DDR4 and an old 40-core Xeon in a Proxmox VM with PCIe passthrough and NUMA binding to one physical CPU.
Prefill is thought to be CPU-limited; ngl, ncmoe and tensor split were manually adjusted to fit the model without crashing.