Local model performance on your hardware

generation:

42.7 tokens/s

prompt processing (prefill):

1294.2 tokens/s

quant:

UD-IQ4_XS (GGUF)

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next at 42.7 t/s generation and 1294.2 t/s prefill on a 126,000-token prompt on an AMD Strix Halo 128GB. Setup is Strata at UD-IQ4_XS with speculative decoding on, one request. tomasreminek's context-131072 request of 2026-10-06. A source-reported measurement, not independently reproduced.

generation:

42.0 tokens/s

prompt processing (prefill):

887.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

int8

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 42.0 t/s generation and 887 t/s prefill on a GMKtec EVO-X2 with AMD Ryzen AI Max+ 395 (Strix Halo, 128GB unified, 64GB GPU carve-out). Setup is Strata 0.1.40.2 (HIP/ROCm) with Unsloth UD-IQ4_XS GGUF, int8 KV cache, 32,768 resident KV tokens, --spec 4 with MTP enabled, and a fresh 17,844-token prompt generating 300 tokens. Single-machine community self-report, not independently reproduced; MTP/speculative decoding is on, so this is not a non-speculative baseline. Chip power was about 121 W (peaks 138 W) and GPU clock median 2.75 GHz.

generation:

53.8 tokens/s

prompt processing (prefill):

1293.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 53.8 t/s output and 1293 t/s prompt processing on Strix Halo 128GB. Setup is Strata with UD-IQ4_XS weights at 8K context; the release adds official Strix Halo support, marked experimental and Linux only. At 64K the run gives 46.6 t/s output and 1370 t/s prompt processing, and at 128K 51.4 t/s output and 1320 t/s prompt processing. The user says it can go up to 1M context length without big speed loss.