Qwen3.6 35B (3B active)
on 8GB VRAM · llama.cpp
Summary
User reports llama.cpp PR #26563, which caches hot MoE experts on GPU, running Qwen3.6-35B-A3B on 8 GB of VRAM at 33.25 to 56.0 t/s with Q2_M and 17.34 to 35.93 t/s with Q5_K_P.
Setup uses autofit with --expert-hot-s -1.
The gains are 1.68x and 2.07x respectively. The user reports negative results for Qwen3.5-122B-A10B and Laguna-S-2.1. The change is CUDA only and applies to single-token decoding only.