NVIDIA · 8GB · 5 reports
Use the calculator to check which models fit in 8 GB of VRAM. The reports below include different GPU counts and memory setups; a model listed here may need extra cards or CPU offloading.
How to compare these reports →CMP 170HX · llama.cpp · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Tested on 4x CMP 170HX 8GB cards flashed to 64GB each (256GB total). Qwen3.6-35B-A3B with MTP, little-MoE default. Generation 63 t/s (110 with MTP optimistic).
4× CMP 170HX · llama.cpp · 1,000,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Tested multiple models on 4x CMP 170HX 64GB cards. DeepSeek V4-Flash 0731 with Q4_K_XL quant, 13B active MoE, 1M context, plain no-spec. Also tested gpt-oss-120B, Qwen3.6-35B-A3B, GLM-4.5-Air, MiniMax-M2.7, and others. Cards are cut-down A100 mining cards with 64GB each, PCIe Gen2 x4, no NVLink.
2× CMP 170HX
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User built a DIY CUDA box with two CMP 170HX cards. Running Qwen Flash Next, reports prompt processing ~4000 tps and decode 80+ tps on a single stream. Also mentions running Qwen 27B and Gemma 26B models.
CMP 170HX · vLLM
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Unlocked CMP 170HX (A100 silicon) with firmware unlock. Tensor throughput 6.3 → 193 TFLOPS. llama.cpp pp512 599.6 → 3468 tok/s. Serving Qwen3.8-27B-FP8 under vLLM, one 170HX beats 2x3090 TP pair on prefill at 197W vs 454W.
CMP 170HX · Ninfer · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
Forked Ninfer for CMP170HX, doubled performance vs llama.cpp on Qwen3.6-35B-A3B. Used for Home Assistant voice assistant.