llamaperf

NVIDIA CMP 170HX 64GB (unlocked)

NVIDIA · 64GB · 15 reports

Is this your card?

On the NVIDIA CMP 170HX 64GB (unlocked), Qwen3.8 27B at 8-bit, the model most run on it, generates a median of 50.0 tokens per second from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the NVIDIA CMP 170HX 64GB (unlocked), each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

Bars are medians on one scale, 0 to 116.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: vLLM 7 (reports) · llama.cpp 3 (reports) · ExLlamaV3 1 (reports) · Strata 1 (reports)

Run models on your NVIDIA CMP 170HX 64GB (unlocked)? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA CMP 170HX 64GB (unlocked)

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 64 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: positiveNVIDIA hardware
generation:
55-65 tokens/s
prompt processing (prefill):
900-1100 tokens/s
quant:
IQ2_XXS GSQ-RCO-GGUF (GGUF)

User reports DeepSeek V4.1 Flash running at 55-65 t/s decode and 900-1100 t/s prefill on a single unlocked CMP 170HX 64GB at 150W. Setup is Project Maya with an IQ2_XXS GSQ-RCO-GGUF (2.0 bpw) quant. Both figures are given as ranges. The user plans to retry at 250-300W once a server shroud arrives.

Nov 2, 2026
Tone: mixedNVIDIA hardware
generation:
25-33 tokens/s
prompt processing (prefill):
900-1100 tokens/s
quant:
IQ2_XXS (GGUF)

User reports DeepSeek V4.1 Flash at around 25-33 t/s decode and 900-1100 t/s prefill at 128-256K context on an unlocked CMP 170HX 64GB, with prefill dropping to 520 t/s at 1M context. Setup is Strata (Project Maya) with an IQ2_XXS GSQ-RCO GGUF at 2.0 bpw, on a Dell R740 server with 256GB DDR4-2133 ECC RAM, GPU capped at 150W and running at 85C without a fan. User asks how others run this card with this model, and says the low-bit DeepSeek V4.1 Flash beats the Qwen3.8-Flash-Next Q6 they ran before on quality, though they avoid expert-preference optimizations that trade quality for speed.

Nov 1, 2026
NVIDIA hardware
generation:
38.0 tokens/s
prompt processing (prefill):
907.0 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3.6 27B at 38 t/s generation on an unlocked NVIDIA CMP 170HX 64GB. Setup is llama.cpp with Q4_K_M and flash attention, full GPU layer offload, zero context depth. Prompt processing measured 907 t/s at pp512 and 888 t/s at pp8192, five repetitions each; the model file is about 16GB.

Oct 14, 2026
Tone: positiveNVIDIA hardware
generation:
100.0 tokens/s
quant:
Q4_K_XL (GGUF)
agentictool-use

User reports Qwen3.8 Flash-Next at 100 tok/s on a CMP 170HX 64GB and one RTX 3090, with system RAM offload. Setup is Strata with Q4_K_XL weights; the run used two GPUs plus system RAM, and the user notes about a quarter of the results came from Strata .38, an eighth from .39, and the rest from .40. Before Strata, the same model ran on three GPUs (CMP 170HX and two RTX 3090s) under Unsloth Studio at 39 tok/s with 84 s average TTFT; the Strata run cut average TTFT to 15.7 s and total LLM time from 5,795m 9s to 1,228m 53s.

Oct 9, 2026
Tone: positiveNVIDIA hardware
generation:
95.6 tokens/s
prompt processing (prefill):
1529.0 tokens/s
quant:
3.05bpw (EXL3)
kv:
Q8
codingagenticlong-context

User reports GLM-5.3-Flash at 95.6 t/s decode and 1,529 t/s prefill on an 8K input, running on two CMP 170HX 64GB cards. Setup is ExLlamaV3 1.5.4 with an EXL3 3.05bpw target, DFlash2 EXL3 6bpw K7 drafter and Q8 KV cache, target weights fully resident in HBM at about 116.6GiB across the two cards. Decode stays near 90 t/s out to a 385K input (90.1 t/s, 1,535 t/s prefill); DFlash acceptance was around 87%, against roughly 60 t/s on the older MTP d2 path. The user also compared a Qwen3.8-Flash-Next AWQ INT4 + FP8 PLE vLLM setup on the same machine, which reached 129.5 t/s decode and 5,513 t/s prefill at 8K, but GLM finished three one-shot coding tasks sooner (4m40s vs 5m30s, 10m05s vs 14m40s, 4m40s vs 24m10s).

Oct 8, 2026
NVIDIA hardware
generation:
117.0 tokens/s
prompt processing (prefill):
6066.0 tokens/s
kv:
fp8

User reports DeepSeek-V4.1-Flash at 117 tok/s decode on one stream at 128k context and 6,066 tok/s prefill at 105k tokens on eight 64 GB CMP 170HX mining cards. Setup is vLLM with fp8 KV cache, PP=8, DSpark speculative decoding with 5 draft tokens, Engram tables in pinned host RAM, and 1M context. Decode drops to 96 tok/s at 512k. Eight concurrent streams reach 532 tok/s aggregate (66 per stream) at 105k. KV pool holds 6.17M tokens. One card was capped to 180 W after PCIe bus drops.

Oct 5, 2026
Tone: positiveNVIDIA hardware
generation:
101.3 tokens/s
prompt processing (prefill):
3191.0 tokens/s
quant:
W4A16 (W4A16)
kv:
BF16
codinglong-contexttool-usemultilingual

User reports Qwen3.8-Flash-Next at 101.3 t/s decode on Japanese prose and 170.0 t/s on code, single stream, on two NVIDIA CMP 170HX cards. Setup is vLLM with W4A16 weights, BF16 KV cache, MTP k=4 speculative decoding, 262,144-token context, and expert parallel across the two cards. The PLE n-gram table was converted to FP8 locally to fit in 92 GiB of host RAM. Aggregate throughput reaches 392 t/s at 4 concurrent requests. Prefill measures 3,191 t/s at 6,954 tokens and 3,261 t/s at 27,853 tokens. A 200,087-token prompt returned the planted code with 70.5 s time to first token and 69.0 t/s decode at depth. MTP is worth about 1.6x on prose and 2.6x on code. The user notes xhigh reasoning effort spent the whole budget without answering in 5 of 6 runs.

Sep 27, 2026
NVIDIA hardware
generation:
95.0 tokens/s
quant:
FP8
kv:
FP8
codinglong-context

User reports DeepSeek V4 Flash Vision at up to ~95 tok/s on 4× CMP 170HX 64GB (256GB aggregate HBM2e). Setup is vLLM with pipeline parallel 4, max_model_len 262144, FP8 KV cache, DSpark speculative decoding with 6 tokens, and max_num_seqs 2. Normal coding workload runs ~50–70 tok/s, strong speculative-decoding periods ~80–90 tok/s, peak ~95 tok/s. Cards sit around 56–62GB VRAM each at 50–60°C. Next tests planned for Qwen3.8 Flash Next and GLM 5.3.

Sep 24, 2026
NVIDIA hardware
generation:
116.0 tokens/s
prompt processing (prefill):
6800-7950 tokens/s
quant:
W4A16

User reports Qwen3-Next-80B at 116 t/s single-stream decode on a single NVIDIA CMP 170HX with 64 GB HBM2e, at a 150 W cap. Setup is vLLM 0.27.1 with W4A16 weights (40.9 GB), torch 2.13.0+cu130, CUDA 13.0, Ubuntu 26.04 LTS, on an AMD Ryzen Threadripper PRO 3945WX with 128 GB DDR4 ECC. Prefill over about 8.9k tokens measured 6800 to 7950 t/s; power draw 137 to 145 W. Aggregate throughput at 8 concurrent requests was 352 t/s, saturating at 4 slots. Also measured on the same card for context: Ornith-1.5-35B FP8 at 122.5 t/s and Qwen3.8-27B W4A16 with DFlash2 at 127 t/s single-stream.

Sep 23, 2026
NVIDIA hardware
generation:
98.1 tokens/s
prompt processing (prefill):
5300.0 tokens/s
quant:
MXFP4+FP8

User reports DeepSeek-V4-Flash-0731 at 98.1 t/s single-stream decode on 4x CMP 170HX (64 GB HBM2e each), up from a 50.8 t/s baseline with DSpark speculative decoding. Setup is vLLM with pipeline parallelism, native MXFP4+FP8 weights (~155.4 GiB), context verified to 1,047,736 tokens. At 100k context single-stream decode is 38.8 t/s with 14.6 s time to first token. Aggregate throughput across 64 concurrent requests is 712.8 t/s with DSpark (472.0 t/s baseline), and 90.0 t/s at 100k context. Prefill across the 25k-77k context range is about 5,300 t/s.

Sep 23, 2026
Tone: positiveNVIDIA hardware
generation:
147.0 tokens/s
quant:
W4A16 (W4A16)
kv:
FP8
long-context

User reports Qwen3.8-27B at 147.0 t/s single-stream decode on one NVIDIA CMP 170HX 64GB, with 134.7 t/s at 4K, 100.1 t/s at 65K, ~90 t/s at 126K, and 64.9 t/s at 250K context. Setup is vLLM 0.27.1 with a W4A16 target, a DFlash2 W4A16 drafter, FP8 target KV, BF16 draft KV, a custom SM80 split-KV verifier, full CUDA Graph, 35 verifier segments / 140 CTAs, k=3 draft tokens, MAX_SEQS=1, and 1350 MHz / 180W. These are decode-only numbers, not end-to-end throughput including prefill. FP8 beat INT8 at long context (48.3 vs 115.4 ms/iter at 250K). Allowing 2 concurrent long requests made makespan and slowest-request throughput worse at both 126K and 250K.

Sep 20, 2026
Tone: mixedNVIDIA hardware
generation:
50.0 tokens/s
quant:
Q8
coding

User reports Qwen3.8 27B at around 50 t/s on a CMP 170HX, running at Q8 or BF16. The user finds the model hallucinates classes and APIs in 90%+ of answers to domain-specific enterprise Java questions, while DeepSeek and Microsoft Copilot produce working code 99% of the time. The user asks whether Qwen's agentic optimization is responsible and requests tips for a harness, system prompt, or skills to reduce hallucination.

Sep 18, 2026
Tone: positiveNVIDIA hardware
generation:
63.0 tokens/s
prompt processing (prefill):
1700.0 tokens/s
quant:
Q6_K
kv:
F16

User reports Qwen3.6-35B-A3B at 63 t/s on 4x CMP 170HX 8GB cards flashed to 64GB each, 256GB total. Setup uses MTP with little-MoE default. Generation reaches 110 t/s with MTP optimistic.

Sep 7, 2026
NVIDIA hardware
generation:
29.0 tokens/s
prompt processing (prefill):
450.0 tokens/s
quant:
Q4_K_XL

User reports testing multiple models on 4x CMP 170HX 64GB cards, including DeepSeek V4-Flash 0731 with a Q4_K_XL quant, 13B active MoE, and 1M context, plain no-spec. The cards are cut-down A100 mining cards with 64GB each, PCIe Gen2 x4, and no NVLink. Other models tested include gpt-oss-120B, Qwen3.6-35B-A3B, GLM-4.5-Air, and MiniMax-M2.7.

Sep 7, 2026
Tone: positiveNVIDIA hardware
prompt processing (prefill):
3468.0 tokens/s
quant:
FP8

User reports an unlocked CMP 170HX, A100 silicon with a firmware unlock, reaching 193 TFLOPS tensor throughput, up from 6.3 TFLOPS. In llama.cpp, pp512 rises from 599.6 to 3468 tok/s. Serving Qwen3.8-27B-FP8 under vLLM, one 170HX beats a 2x3090 tensor-parallel pair on prefill at 197W versus 454W.

Aug 28, 2026

Get a weekly email of new NVIDIA CMP 170HX 64GB (unlocked) reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3.6 27B
NVIDIA CMP 170HX 64GB (unlocked)
Q4_K_M
llama.cpp
Not reported38.0 tokens/s
Qwen3.8 125B (6B active) Flash-Next
2× NVIDIA CMP 170HX 64GB (unlocked)
Q4_K_XL
Strata
Not reported100.0 tokens/s
GLM-5.3 320B (18B active) Flash
2× NVIDIA CMP 170HX 64GB (unlocked)
3.05bpw
ExLlamaV3
8,19295.6 tokens/s
DeepSeek V4.1 Flash 552B (16B active)
8× NVIDIA CMP 170HX 64GB (unlocked)
Not reported
vLLM
1,048,576117.0 tokens/s
Qwen3.8 125B (6B active) Flash-Next
2× NVIDIA CMP 170HX 64GB (unlocked)
W4A16
vLLM
262,144101.3 tokens/s
DeepSeek V4 Flash Vision
4× NVIDIA CMP 170HX 64GB (unlocked)
FP8
vLLM
262,14495.0 tokens/s

Frequently asked

How fast is the NVIDIA CMP 170HX 64GB (unlocked) for local LLMs?

On the NVIDIA CMP 170HX 64GB (unlocked), Qwen3.8 27B at 8-bit, the model most run on it, generates a median of 50.0 tokens per second from one plain run (community reports, as of 22 Nov 2026). Also reported: Qwen3-Next 80B · 3B active at 4-bit (116.0 t/s, one plain run); Qwen3.6 27B at 4-bit (38.0 t/s, one plain run).

What is the biggest model reported running on the NVIDIA CMP 170HX 64GB (unlocked)?

The largest model with a plain generation run on the NVIDIA CMP 170HX 64GB (unlocked) is Qwen3-Next 80B · 3B active at 4-bit (116.0 t/s, one plain run), as of 22 Nov 2026. Plain means one device, one request, no speculative decoding, so a run split across several cards does not count.