NVIDIA · 16GB · 13 reports
Use the calculator to check which models fit in 16 GB of VRAM. The reports below include different GPU counts and memory setups; a model listed here may need extra cards or CPU offloading.
How to compare these reports →Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Model is a 35B MoE agentic model, beats Qwen3.6 and DeepSeek V4 Pro in some scenarios. Repository: SC117/Agents-A1-Uncensored-MTP-APEX-GGUF. Also mentions a 4B dense variant but not attractive.
RTX 5070 Ti · llama.cpp · 196,608 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports running Qwen3.8 27B at Q4 on 16GB VRAM with 200K context at 50 t/s. They pruned non-ASCII characters from embedding table and LM head to save 700MB, offloaded embedding table to save 270MB, disabled MTP, and used adaptive-kv streaming to fit 196,608 tokens. They used Q8_0 K and Q4_0 V cache quantization. The post is enthusiastic about the setup.
RTX 5070 Ti · llama.cpp · 32,768 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Benchmarked Qwen3.8-27B at Q3_K_XL on RTX 5070 Ti 16GB with llama.cpp. Achieved ~73 tok/s at 32k context with MTP on, fully GPU-resident. Detailed analysis of context ceiling and MTP trade-offs. At-depth generation 64.5 tok/s at 24.1k depth. Prefill 1647 tok/s at depth. MTP off gives ~50 tok/s. Context ceiling at 40k practical, 49k with MTP off. Also compared Q3_K_XL vs Q2_K_XL: Q3 faster than size predicts.
RTX 5070 Ti · llama.cpp · 65,536 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Guide for optimizing Qwen3.8-27B on 16GB VRAM + 64GB system RAM. Uses FFN offloading to keep attention on GPU, q8_0 KV cache at 65k context, MTP speculative decoding, thread pinning. Reports ~18-20 t/s generation speed.
RTX 5070 Ti · llama.cpp · 96,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User tested llama.cpp RPC with a 5070 Ti and 1080 Ti over gigabit ethernet. They found it functional, with trade-offs between prefill and generation speeds. They used Qwen 3.8 27B with UD-Q4_K_XL quant, Q8 KV cache, and 96k context. Best generation speed was 36 t/s with MTP enabled and GPU order RPC0,CUDA0. Prefill-focused config gave 560 pp and 19 tg. They also tested UD-Q5_K_XL with full KV cache: 380 pp and 15 tg. They noted MTP tanks prefill speeds and uses VRAM, and that enabling KV quant for draft model reduces context. They recommend jumbo frames and careful GPU ordering.
2× RTX 5070 Ti · llama.cpp · 49,152 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Benchmarked Qwen3.8-27B on a laptop GPU (RTX 5070 Ti Laptop 12GB) + eGPU (RTX 5060 Ti 16GB) with llama.cpp layer-split. Reported pp512/tg128 for Q4_K_XL: 1003.85/22.30 t/s. Also tested Q5_K_XL (923.21/19.11) and Q6_K (901.18/18.23). MTP speculative decoding boosted generation to 32-38 t/s. Context ceiling found at 49,152 for Q6_K and 73,728 for Q5_K_XL. Notes on tensor-split syntax differences and eGPU power interruption.
3× RTX 5070 Ti · llama.cpp · 256,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports success with AtomicChat build of Qwen3.8-Flash-Next, which splits n-gram table to SSD, allowing full 256K context on 96GB RAM + 40GB VRAM. Previously with unsloth quants, crashes at ~60K context and slower 12-14 t/s. AtomicChat achieves 22 t/s and handles 217K token conversation without crash. Prefill takes ~18 min for 220K context.
RTX 5070 Ti · beellama.cpp · 96,256 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User built a villager simulation game POC with Qwen3.8-27B-UD-Q3_K_XL.gguf on RTX 5070 Ti 16GB, fully offloaded. Vision on CPU. Used beellama.cpp with kvarn performance enhancements. MTP n-max=2, KV=kvarn3/kvarn3, MTP draft KV=kvarn2/kvarn2. Context 96256, tail tokens 1024. tg up to 75 t/s, pp up to 1700 t/s. Lessons: don't fear Q3 quants or KV quantisation; kvarn3 works well. Needle test: Q3_K_XL almost 100% with kvarn3, Q3_XXS does poorly (84% with F16 KV, ~80% with Q3 KV). kvarn better with KLD. Used pi harness with pi-observational-memory, pi-web-access, pi-atelier extensions. No Cursor used. Game features: large map, minimap, zoom, resources, houses, weather/seasons, day/night cycle, possible death, game speed controls, obstacle avoidance, deconstruct buildings. Code quality unknown, but performance surprising. Context management improved with pi-observational-memory. Conclusion: find right balance between model quant and KV quant; for 16GB VRAM Q3_K_XL with kvarn3 works well.
2× RTX 5070 Ti · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports 60 tok/s decode and 1500 pp with Qwen3.8-27B-IQ4-XS-MTP using tensor parallelism across a 5070 Ti and one 5060 Ti. Considering adding a 3090 FE for more VRAM but concerned about mixing Ampere with Blackwell.
2× RTX 5070 Ti · llama.cpp · 220,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Benchmark of Qwen3.8-27B on dual RTX 5070 Ti 16GB. llama.cpp with Q4_K_XL quant, MTP3 speculative decoding, 220k context. Also tested vLLM and NInfer. llama.cpp praised for simplicity and performance.
RTX 5070 Ti · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Used mmap to fit model into 16GB VRAM + 64GB system RAM. Compared to non-MoE 30B model at 10 t/s. CPU: 9800X3D.
RTX 5070 Ti · LM Studio
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Switched from Qwen3.6 35b-a3b IQ4_XS to Qwen3.6 27b IQ3_M. 35b-a3b got 50-60 t/s but slow prompt processing; 27b got ~40 t/s but consistent. 27b found a bug the 35b couldn't. Dense model handles compression better than MoE.
2× RTX 5070 Ti · llama-server · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports using a 5070Ti 16GB and a 2060 6GB to run Qwen3.6-27B Q4_K_M with llama-server. At 71k actual context, pp=186.76 t/s, tg=19.21 t/s. Also provides llama-bench results with CUDA showing tg speeds around 16-25 t/s depending on configuration.