Local model performance on your hardware

generation:

53.3 tokens/s

prompt processing (prefill):

420.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 53.3 tok/s per-request decode (47 to 58 across eight requests) on an Intel Arc Pro B60 24GB. Setup is llama.cpp Vulkan build b11524 with Q4_K_M and an F16 KV cache, a DFlash2 draft model, one request at a time, 32K context, the whole model on the card. The repo's 42.3 tok/s headline is wall-clock eval-suite throughput with prompt processing included, not a decode rate; stock llama.cpp with no drafter gave 16 tok/s in llama-bench tg32. Prompt reading at 32K was 420 tok/s with the Intel FA kernel, versus 118 tok/s stock.

generation:

91.9 tokens/s

prompt processing (prefill):

1760.0 tokens/s

quant:

Q4_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Nemotron 3.5 Lightning 30B-A3B at 91.91 tok/s decode on 2x Intel Arc Pro B60 24GB. Setup is llama.cpp SYCL with Q4_0 weights and an MTP Q8_0 drafter at --spec-draft-n-max 7, 22.18 GiB VRAM, 1,760 tok/s prefill at 12K context. MTP acceptance is 99.5-100% at every n-max; the model needs the whole card with no co-residence.

generation:

15.3 tokens/s

prompt processing (prefill):

513.4 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B at 15.34 t/s generation and 513.39 t/s prompt processing on a single Intel Arc Pro B60. Setup is llama.cpp build b10452 with Vulkan backend, Q4_K_M GGUF weights and Q8_0 KV cache, full GPU offload with Flash Attention, at 512 tokens of context. A second run on two Arc Pro B60 cards gives 14.99 t/s decode and 507.93 t/s prefill at 512 tokens; dual-card prefill scales to 643.16 t/s at 1k, 718.37 t/s at 2k and 149.30 t/s at 64k, while decode stays near 15 t/s.

generation:

24.0 tokens/s

prompt processing (prefill):

609.5 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 24.00 tok/s decode on dual Intel Arc Pro B60 24GB (48GB total). Setup is llama.cpp build b11100 with SYCL F16 JIT, Q4_K_M GGUF, Q8_0 KV cache, 131072 context, and native embedded Q8_0 MTP draft speculative decoding. Prefill is 609.47 tok/s at 512 tokens, scaling to 925.39 tok/s at 4k and 570.24 tok/s at 128k. Stock non-MTP decode is 16.13 tok/s; live llama-server chat with MTP ranges 20.00-30.29 tok/s.

generation:

8.3 tokens/s

prompt processing (prefill):

112.6 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.5 35B-A3B at 8.34 t/s generation and 112.62 t/s prompt processing on an Intel Arc Pro B60 24GB. Setup is llama.cpp build 8175 with the SYCL backend, Q4_K_XL GGUF weights, full GPU offload (-ngl 100). User reports the card is ok for chat-length context on smaller models but slow for agentic tools like opencode/claude code, where initial prompt response can take 5-10 minutes. A comparison run of the same model on an RTX Pro 4500 Blackwell reached 133.47 t/s generation and 3807.62 t/s prompt processing.

generation:

22.2 tokens/s

prompt processing (prefill):

245.5 tokens/s

quant:

Q4_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks a Qwen3.8-27B Ridge Intel Arc tuned GGUF on an Intel Arc Pro B60 24GB, reporting 22.23 tok/s pure autoregressive decode and 245.54 tok/s prompt prefill. Setup is llama.cpp with Q4_K weights, -ngl 99 and -fa on, on a single 24GB card. With embedded MTP speculative decoding the tuned Ridge build reaches 41.28 tok/s at 93.4% acceptance, and the DAS Lab IQ3_S tuned build reaches 41.88 tok/s at 92.9% acceptance. Stock IQ3_XXS decodes at 8.10 tok/s and stock Unsloth UD-Q4_K_S at 12.97 tok/s. At ~150k-token depth decode falls to 7.91 tok/s with DFlash2 and 7.21 tok/s with native MTP.