NVIDIA · 48GB · 5 reports
Use the calculator to check which models fit in 48 GB of VRAM. The reports below include different GPU counts and memory setups; a model listed here may need extra cards or CPU offloading.
How to compare these reports →RTX 6000 · vLLM-Moet · 130,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Running DeepSeek V4 Flash on a single RTX 6000 Pro using vLLM-Moet. Context length set to 130K. Requires ~150GB RAM for loading. Benchmarks show generation speeds around 100-130 t/s.
2× RTX 6000 · llama.cpp · 250,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports running DeepSeek V4 Flash 0731 with Q3_K_XL quant on dual GPU setup (RTX 6000 96GB + W7800 48GB) using llama.cpp server. Achieved 27.2 t/s generation speed. Mentions it's cheaper than K3 and GLM 5.2 but very good.
2× RTX 6000 · sglang · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User runs Qwen 3.8 27B FP8 on two RTX 6000 Pro GPUs with sglang, getting 150 tk/sec but tasks take 12x longer than Claude Opus 5. They also tested BF16 on primary card with FP8 offloading, which took 3 hours vs 1h45m for FP8. Context lengths tested: 256k, 128k (too small), 500k (faster despite unsupported). They use a skill that reads large documentation files, hitting 80k context before starting. They compare against Claude Code, Pi Code, Qwen Code, and Hermes. They are learning and using ChatGPT 5.6 high for testing.
RTX 6000 · ik_llama · 200,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports prefill ~2000 tps and decode ~40 tps for single request; with 2-4 parallel requests, prefill drops to 500-1000 tps and decode to 10-20 tps. They are seeking a better setup for 2-4 slots with q4 quants or better and q8 kv cache or better. They mention trying vLLM recipes before.
RTX 6000 · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports dflash2 speeds up Qwen 3.8 27B up to 4x, with median 140.6 tok/s vs baseline 47.4 tok/s. Caveat: gains vary by task, sometimes only 1.5x. Mentions being from atomic.chat team.