NVIDIA CMP 170HX 8GB reports

generation:

121.0 tokens/s

prompt processing (prefill):

630.0 tokens/s

quant:

IQ3_XXS (GGUF)

kv:

int8

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8 Flash-Next at 121 t/s generation and 630 t/s prefill on two CMP 170HX 8GB cards. Setup is Strata with IQ3_XXS GGUF weights, int8 KV cache, MTP speculative decoding with 4 tokens, and a 262144 max context. The table covers prompt lengths from ~0.5K to 140K and concurrency 1 to 4; the reported figures are the c1 ~0.5K row. Total output tok/s at c1 ~0.5K is 89.