llamaperf

Qwen3.6 35B (3B active)

on NVIDIA V100 16GB · AgrillaMoE

Tone: positiveNVIDIA hardware
Oct 9, 2026
Throughput
57-60 t/s gen
Quant
UD-Q2_K_XL (GGUF)
VRAM reported
16 GB

Use cases

coding

Summary

User reports Qwen3.6-35B-A3B at 57-60 tok/s on a rented V100 16GB. Setup is AgrillaMoE, a fork of llama.cpp's server, with the 2-bit UD-Q2_K_XL quant and a MoE-expansion patch raising active experts to 20 on layers 25-39. The user says the expansion patch improves GPQA-Diamond from 81.82% to 84.34% with Q8_0, and that the server speaks both OpenAI and Anthropic APIs.