Qwen3.8 2400B (95B active)
on 2× NVIDIA RTX 5090 · llama.cpp · 512 ctx
Sep 7, 2026
Summary
User reports Qwen3.8-2.4T-A95B at ~0.80 tok/s generation on an RTX 5090 plus an RTX 5060 Ti.
Setup is llama.cpp (Unsloth build 10360) with the Unsloth Q1_0 GGUF (397 GiB), native MTP speculative decoding at n_max=3 and p_min=0.5, and 92 experts offloaded to CPU.
MTP acceptance is 90.48%, a +3.64% throughput gain over no MTP. VRAM use is 29.6 GB on the 5090 and 12.3 GB on the 5060 Ti.