llamaperf

Qwen3.6 35B (3B active)

on NVIDIA RTX 5060 Ti 16GB · llama.cpp · 51,200 ctx

Tone: positiveNVIDIA hardware
Sep 7, 2026
Throughput
40.0 t/s gen
Quant
Q4_K_XL (GGUF)
KV cache
q8_0
System RAM
16 GB
VRAM reported
16 GB

Use cases

vision

Summary

User reports Qwen3.6-35B-A3B MoE at about 40 t/s on an RTX 5060 Ti 16GB with mmproj enabled, and just over 50 t/s without mmproj and with --n-cpu-moe 24. Setup is llama.cpp server in Docker with --cache-type-k/v q8_0, --cache-ram 0, --no-mmap, and --n-cpu-moe 19, or 24 without vision. The user experienced an OOM shutdown before adding --cache-ram 0 and is seeking advice on the setup.