Qwen3.6 35B (3B active)

User reports Qwen3.6-35B-A3B at AWQ 4-bit decoding at about 120 tokens/second under a single request on 2x modded 22GB RTX 2080 Ti with NVLink. Setup is vLLM with tensor parallelism across the two cards. The post's main benchmark ran a W8A8-INT8 build with a TurboQuant KV cache (about 395K tokens) and up to 16 concurrent requests, but those results appear only in screenshots; the author notes AWQ 4-bit is the better choice on Turing.