Local model performance on your hardware

generation:

38.0 tokens/s

prompt processing (prefill):

907.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

LTT Labs original tests, September 12, 2026. Selected run: one unlocked NVIDIA CMP 170HX, 64GB GPU memory, Qwen 3.6 27B Q4_K_M GGUF. Prefill pp512: 907 tokens/s; pp8192: 888 tokens/s; generation tg128: 38 tokens/s. Command: ./llama-bench -m <model> -fa 1 -ngl 999 -r 5 -d 0 -n 128 -p 512,8192. All tests have zero context depth; five repetitions, flash attention and full GPU layer request. CPU, system RAM, runtime commit, CUDA version, actual resident memory and tested artifact revision are unknown. Approximately 16GB is model file size, not resident RAM. Exact tested quantization repository linked by author: https://huggingface.co/bartowski/Qwen_Qwen3.6-27B-GGUF . Source: https://www.lttlabs.com/articles/2026/09/12/cmp-170hx-cmpunlocker . Other models and multi-card comparisons in the source are separate runs.

generation:

63.0 tokens/s

prompt processing (prefill):

1700.0 tokens/s

quant:

Q6_K

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 63 t/s on 4x CMP 170HX 8GB cards flashed to 64GB each, 256GB total. Setup uses MTP with little-MoE default. Generation reaches 110 t/s with MTP optimistic.

generation:

29.0 tokens/s

prompt processing (prefill):

450.0 tokens/s

quant:

Q4_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports testing multiple models on 4x CMP 170HX 64GB cards, including DeepSeek V4-Flash 0731 with a Q4_K_XL quant, 13B active MoE, and 1M context, plain no-spec. The cards are cut-down A100 mining cards with 64GB each, PCIe Gen2 x4, and no NVLink. Other models tested include gpt-oss-120B, Qwen3.6-35B-A3B, GLM-4.5-Air, and MiniMax-M2.7.