Qwen3.8 27B
on 2× NVIDIA RTX 3060 12GB · llama.cpp · 131,072 ctx
Oct 5, 2026
Summary
User reports Qwen3.8-27B at 56.0 tok/s decode and 497 tok/s prefill on 2× RTX 3060 12GB.
Setup is llama.cpp CUDA with UD-Q4_K_XL GGUF, q4_0 KV cache, 131072 context, tensor split across the two cards, and MTP speculative decoding. The 106,294-token context gate took 213.8 s.
A second recipe runs Qwen3.6-35B-A3B on the same machine at 72.6 tok/s decode and 512 tok/s prefill with q8_0 KV cache and layer split, keeping some expert tensors in system RAM (8.22 GiB host RAM drop).