Qwen3.8 reports

generation:

36.0 tokens/s

prompt processing (prefill):

340.0 tokens/s

quant:

QFN Q_2

kv:

q4_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next at 36 t/s generation and 340 t/s prefill on 8GB VRAM and 32GB RAM. Setup is Strata with QFN Q_2 quant and q4_0 KV cache, 176128 context, 2000 experts cached, MTP disabled. User says this is the best model they have run on this hardware, with no tool call issues or looping, and ran past 3 compactions at 192k context with fewer cached experts.

generation:

30-38 tokens/s

prompt processing (prefill):

950-1800 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next running at 30-38 tok/s output and 950-1800 tok/s prefill on an RTX 4070 Super 12GB with 32GB system RAM. Setup is Strata with 256K context, MTP and vision off, model partially offloaded to system RAM. User notes speeds depend heavily on CPU, SSD, RAM speed and GPU bandwidth, and that 16GB VRAM may allow MTP.

generation:

33.5 tokens/s

quant:

UD-Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 33.5 t/s generation. Source evidence (verbatim): Source: https://llmkube.com/docs/labs/qwen38-flash-next-gufo-on-strix-halo # First-hand Gufo Qwen3.8 Flash Next UD-Q4_K_XL single-stream generation on Strix Halo 128GB Christopher Maher's first-hand lab report, live deployment dated October 2, 2026. Measured AMD Ryzen AI Max+ 395 / Radeon 8060S gfx1151, Strix Halo 128GB. Installed unified memory: 128GB LPDDR5X. Selected single-stream agent-style run: generation 33.5 tokens/s, using MTP. Approximately 4K tokens of Go source plus coding instruction; up to 512 completion tokens; reasoning included, thinking defaults on. Server has two session slots, but this measured row has one stream, not two concurrent requests. Decode is completion tokens divided by generation time. Selected run prompt-processing speed unreported; leave promptTps null. Separate cold-prefill and two-stream rows must not be combined with this selected generation run. Qwen3.8-Flash-Next UD-Q4_K_XL, GGUF. Tested repository verified: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF ; source reports short revision 38bb39ee, independently opened revision tree returned a generic tool error; full SHA not verified. Four quantization shards, 103.69 GiB on disk. Shared Q8_0 MTP head path: MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf in that repository; 2.60 GiB disk size. Disk sizes are not resident RAM. Gufo HIP engine, context capacity 262144. Exact commit/version for selected bake-off row unestablished; later deployed Gufo v0.5.0/23cacbb9379d5a8e8531535270891729ca6c00dd is separate and must not be assigned to this earlier measurement. Core counts, power, selected resident memory, exact prompt count, CPU offload, KV quantization and sampling settings unreported. Lab author builds LLMKube. Client uses workstation kubectl port-forward to an otherwise idle node, one candidate at a time. Sources: https://llmkube.com/docs/labs/qwen38-flash-next-gufo-on-strix-halo ; https://llmkube.com/blog/qwen38-flash-next-gufo-strix-halo .

generation:

12.1 tokens/s

prompt processing (prefill):

378.2 tokens/s

quant:

UD-Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 12.1 t/s generation and 378.2 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama-bench on the ROCm backend with the UD-Q4_K_XL GGUF, no speculative decoding, no chat template and no client, at depth 0. The run is a depth sweep: generation falls from 12.06 t/s at depth 0 to 9.05 t/s at depth 100000, and prompt processing from 378.24 t/s to 121.85 t/s. Each PP and TG entry is a separate phase, not one request.

generation:

7.5 tokens/s

prompt processing (prefill):

236.7 tokens/s

quant:

Q8_0 (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 7.5 t/s generation and 236.7 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with official Q8_0, full offload (ngl 999), batch 2048, flash attention and 32 threads. The figures are separate llama-bench pp512 and tg128 microbenchmarks, not one end-to-end request; the home narrative reports 7.53 t/s for the same run.

generation:

19.6 tokens/s

prompt processing (prefill):

658.5 tokens/s

quant:

UD-IQ3_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B UD-IQ3_XXS at 19.6 t/s generation and 658.5 t/s prompt processing on one NVIDIA RTX 3060 12 GB plus one Radeon 8060S integrated in a Ryzen AI Max+ 395. Setup is llama.cpp native llama-bench with heterogeneous CUDA and Vulkan layer stages across the two GPUs. The headline prompt and generation figures come from separate tests, not a paired 4096-input/64-output request; concurrency was 1, and the source reports 27.32B parameters, 10.17 GiB model size and 3.06 bpw.

generation:

18.3 tokens/s

prompt processing (prefill):

313.3 tokens/s

quant:

UD-IQ3_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B UD-IQ3_XXS at 18.3 t/s generation and 313.3 t/s prompt processing on a single Ryzen AI Max+ 395 / Radeon 8060S. Setup is llama.cpp with llama-bench over Vulkan; the RTX 3060 in the same host is excluded from this control. The 313.3 t/s is a pp4096 test and the 18.3 t/s a tg64 test, independent synthetic runs rather than a paired 4096-input/64-output request. A pp65536 test reached 136.69 t/s, and a pp98304 test timed out after 900 seconds.

generation:

15.7 tokens/s

prompt processing (prefill):

255.7 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 15.7 t/s generation and 255.7 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with Q8_0 GGUF, flash attention, F16 K/V cache, and requested GPU layers 99 on one Metal device. The headline is depth 0: PP512 255.725 and TG128 15.697 tokens/s, separate 512-token prefill and 128-token generation microbenchmarks rather than an end-to-end conversation. At prefix depth 16384, generation falls to 14.931 t/s.

generation:

17.2 tokens/s

prompt processing (prefill):

246.3 tokens/s

quant:

UD-Q6_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 17.2 t/s generation and 246.3 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q6_K_XL GGUF, flash attention, F16 K/V, 99 GPU layers requested, single sequence. Figures are depth-0 llama-bench PP512 and TG128 microbenchmarks, not end-to-end; at prefix depths 4096 and 16384 generation falls to 16.94 and 16.29 t/s.

generation:

19.7 tokens/s

prompt processing (prefill):

233.5 tokens/s

quant:

UD-Q5_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 19.7 t/s generation and 233.5 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q5_K_XL GGUF, F16 K/V cache, flash attention, and requested GPU layers 99 on one Metal device. The headline is depth 0; at prefix depth 4096 generation is 19.45 t/s and at 16384 it is 18.54 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

21.8 tokens/s

prompt processing (prefill):

236.6 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 21.8 t/s generation and 236.6 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q4_K_XL GGUF, F16 K/V cache, flash attention, and 99 GPU layers requested on one Metal device. Figures are depth-0 llama-bench microbenchmarks: PP512 at 236.56 t/s and TG128 at 21.76 t/s, single sequence, five repetitions. At prefix depths 4096 and 16384, generation falls to 21.52 and 20.41 t/s.

generation:

23.5 tokens/s

prompt processing (prefill):

249.0 tokens/s

quant:

UD-Q3_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 23.5 t/s generation and 249 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q3_K_XL GGUF, flash attention, F16 K/V, and 99 GPU layers requested on a single Metal device. The headline is depth 0; at prefix depths 4096 and 16384 generation falls to 23.07 and 21.77 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

24.8 tokens/s

prompt processing (prefill):

251.1 tokens/s

quant:

UD-Q2_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 24.8 t/s generation and 251.1 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q2_K_XL GGUF, flash attention, F16 K/V, 99 GPU layers requested, single sequence, five repetitions. Headline is depth 0: PP512 251.10 and TG128 24.76 tokens/s. At prefix depth 4096 generation falls to 24.23 t/s and at 16384 to 22.83 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

24.2 tokens/s

prompt processing (prefill):

250.4 tokens/s

quant:

UD-IQ3_XXS (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 24.2 t/s generation and 250.4 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-IQ3_XXS GGUF, F16 K/V cache, flash attention, 99 GPU layers requested, single sequence. Headline is depth 0; at prefix depth 4096 generation is 23.81 t/s and at 16384 it is 22.34 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

24.2 tokens/s

prompt processing (prefill):

251.7 tokens/s

quant:

UD-IQ2_XXS (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 24.2 t/s generation and 251.7 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-IQ2_XXS GGUF, F16 K/V cache, flash attention, and 99 GPU layers requested on a single Metal device. The headline is depth 0; at prefix depths 4096 and 16384 generation falls to 23.79 and 22.40 t/s. These are separate 512-token prefill and 128-token generation microbenchmarks, not an end-to-end conversation.

generation:

49.5 tokens/s

prompt processing (prefill):

358.5 tokens/s

quant:

W4B HGN + quality overlay

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 at 49.5 t/s generation and 358.5 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is Halogen 0.9.1 with a W4B HGN quant plus quality overlay, 262144 context per request, MTP and prompt lookup enabled, on one Ryzen AI MAX+ 395 APU with 128 GB unified memory. The 49.5 t/s and 358.5 t/s come from a single idle-peer fixed-code repeat with 524 input and 256 output tokens; the same run's client-side generation was 50.974291±1.201111 t/s.

generation:

6.5 tokens/s

prompt processing (prefill):

60.8 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

F16

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 6.5 t/s generation and 60.8 t/s prompt processing on a Mac mini M4 with 24 GB unified memory. Setup is llama.cpp build 9544 with the UD-IQ4_XS GGUF, run via llama-bench with -p 512 -n 128, F16 K/V cache and flash attention AUTO. The source's native figures are PP512 = 60.78 +/- 1.60 tokens/s and TG128 = 6.47 +/- 0.03 tokens/s, separate prompt and generation microbenchmarks rather than one request; TG128 is depth-0 generation, not decode after a 512-token prompt.

generation:

36.8 tokens/s

prompt processing (prefill):

284.5 tokens/s

quant:

Q4_0_ROCMFP4_FAST (GGUF)

kv:

Q8

flash attention:

on

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 36.8 t/s generation and 284.5 t/s prefill on an AMD Strix Halo 128GB box. Setup is the ROCmFPX llama.cpp fork with Q4_0_ROCMFP4_FAST, q8_0 K and V cache, flash attention on, one parallel slot, 131072 context, and native MTP with max 4 draft tokens. The 36.8 t/s is the median of four prompt categories at temp 0, single stream with 600-token outputs; the 284.5 t/s prefill is a separate 11,626-token prompt measurement.

generation:

12.2 tokens/s

prompt processing (prefill):

291.6 tokens/s

quant:

Q4_K_XL (GGUF)

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 12.2 t/s generation and 291.6 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 434ddbb with the Q4_K_XL quant, Vulkan backend, single Radeon 8060S GPU, and a 1GB BIOS carve plus 120GB GTT region. The figures are separate synthetic PP512 and TG128 tests, not a 512-input/128-output request; the source means are 291.62 +/- 28.60 t/s and 12.15 +/- 0.02 t/s over 5 repeats.

generation:

7.8 tokens/s

prompt processing (prefill):

278.8 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B Q8_0 at 7.8 t/s generation and 278.8 t/s prompt processing on an AMD Ryzen AI Max+ 395 / Radeon 8060S with 128 GiB unified RAM. Setup is llama.cpp build 10603 with Vulkan RADV/Mesa 26.1.7, -ngl 999 -b 2048 -ub 512 -fa on, F16 K/V, one GPU, 5 repetitions. The headline pairs same-backend Vulkan pp512 278.80 ±9.40 t/s with tg128 7.77 ±0.00 t/s; the reported SD of 0.00 is rounded, not proof of zero variability. TG128 is 128 sequential synchronized random-token decode work units, not delivered text, and the PP sizes are timed synthetic work, not maximum context capacity.