Qwen3.6 35B (3B active)
on NVIDIA V100 16GB · AgrillaMoE
Use cases
coding
Summary
User reports Qwen3.6-35B-A3B at 57-60 tok/s on a rented V100 16GB.
Setup is AgrillaMoE, a fork of llama.cpp's server, with the 2-bit UD-Q2_K_XL quant and a MoE-expansion patch raising active experts to 20 on layers 25-39.
The user says the expansion patch improves GPQA-Diamond from 81.82% to 84.34% with Q8_0, and that the server speaks both OpenAI and Anthropic APIs.