Local model performance on your hardware

generation:

24.6 tokens/s

prompt processing (prefill):

492.0 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Swift-Qwen3.8-27B at 24.6 t/s decode and 492 t/s prefill on a single Radeon PRO V620 32GB. Setup is a llama.cpp fork for gfx1030 with Q4_K_XL GGUF (16.6 GiB), flash attention, 128k context, card at 300 W. Decode falls to 16.9 t/s at 128k depth; with an MTP drafter it reaches 48.0 t/s at depth 0 and 44.1 t/s at 120k, while DFlash2 peaks at 76 t/s on short mixed work.

generation:

35.4 tokens/s

prompt processing (prefill):

355.4 tokens/s

quant:

Q6_K_XL (GGUF)

kv:

f16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Muse Glimmer 30B on AMD v620 GPUs, with Q6 on one GPU at 355.38 t/s prefill and 35.38 t/s generation. Setup is a tensor split across two GPUs: Q6 gives 472.27 t/s prefill and 36.32 t/s generation, Q8 gives 550.04 t/s prefill and 26.55 t/s generation. Speculative decoding uses a dflash draft model.

generation:

21.1 tokens/s

prompt processing (prefill):

276.2 tokens/s

quant:

IQ3_XXS

kv:

FP16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek-V4-Flash-0731 at 21.06 t/s continuous generation and 30.59 t/s sustained accepted generation on 4x AMD Radeon Pro V620 with 128 GB total. Setup is the unsloth IQ3_XXS quant with speculative decoding using a DSpark Q8 drafter on CPU RAM and 2 draft tokens. Prompt ingestion is 276.23 t/s at 32K and 379.01 t/s at 4K. The user calls the model at q3_xxs "fucking stupid" and lobotomized, and notes there is not enough VRAM for the drafter, which affects TG speeds.

generation:

14.3 tokens/s

prompt processing (prefill):

364.0 tokens/s

quant:

Q5_K_P (GGUF)

kv:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B and Gemma4-26B-A4B on an AMD V620 using llama.cpp with ROCm and Vulkan. Qwen results are shown; Gemma results appear in a table but are not extracted as the primary model.