AMD Radeon Pro V620 32GB reports

generation:

64.0 tokens/s

prompt processing (prefill):

1200.0 tokens/s

quant:

AWQ W4A16 (AWQ)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at ~64 tokens/s single-stream decode and ~1,200 tokens/s prefill on four AMD Radeon PRO V620 cards (32 GB each, gfx1030) in one box. Setup is a vLLM fork with AWQ W4A16 backbone plus a quantised int4 n-gram table, tensor parallel across all four cards, MTP off and cards uncapped; the n-gram table is CPU-offloaded. At the production 140 W setting decode is ~59.5 tokens/s, and with MTP=3 it ranges ~60-72 tokens/s. Model files total ~105 GB of downloads.

generation:

24.6 tokens/s

prompt processing (prefill):

492.0 tokens/s

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Swift-Qwen3.8-27B at 24.6 t/s decode and 492 t/s prefill on a single Radeon PRO V620 32GB. Setup is a llama.cpp fork for gfx1030 with Q4_K_XL GGUF (16.6 GiB), flash attention, 128k context, card at 300 W. Decode falls to 16.9 t/s at 128k depth; with an MTP drafter it reaches 48.0 t/s at depth 0 and 44.1 t/s at 120k, while DFlash2 peaks at 76 t/s on short mixed work.

generation:

42.0 tokens/s

quant:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen 3.8 27B at up to 42 t/s on 2x Radeon Pro V620 32GB cards at 41k context. Setup is vLLM 0.27.1 with a Q4 quant and tensor parallelism of 2. The first working run gave 11-18 t/s; after optimization the two-card combo reached 42 t/s. A four-card TP=4 config for a 122B MoE reaches 56-59 t/s. A successor repo runs Qwen3.8-Flash-Next at ~100 t/s single-stream on 4x V620, versus 29-30 t/s for llama.cpp on the same box.

generation:

62-68 tokens/s

quant:

AWQ-W4A16 (AWQ)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 62-68 tok/s decode on 3x Radeon PRO V620 32GB in pipeline parallel PP=3. Setup is vLLM with an AWQ-W4A16 checkpoint, int4 n-gram sidecar in host RAM, MTP k=2 speculative decoding, 262K context, 160 W power cap per card. Prefill measured at 1152 / 1705 / 1936 / 1933 tok/s for 4K / 16K / 64K / 130K prompts; a 200 W cap raises prefill to 1306 / 1943 / 2221 / 2254 tok/s with decode unchanged. Multi-stream decode without MTP reaches 157 tok/s at 4 streams and 242 at 8; with MTP 159 at 4 and 91 at 8.

generation:

60-100 tokens/s

quant:

W4A16 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 60 to 100 t/s decode and over 3000 t/s prefill on a rig with 8x Radeon Pro V620 (256 GB VRAM). Setup is a custom vLLM fork with W4A16 routed experts and BF16 elsewhere, pipeline parallel 4, MTP enabled with 3 token drafting, concurrency 1. User says llama.cpp managed only around 350-450 t/s prefill with the same model, and that the new setup is about 800% faster.

generation:

70.0 tokens/s

prompt processing (prefill):

2000.0 tokens/s

quant:

W4A16 (AutoRound)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at roughly 2,000 t/s prefill and 70 t/s decode on 4x Radeon Pro V620 32GB. Setup is Intel Autoround W4A16 weights running locally with the OMP harness, which the user believes made a big difference. The model ran for around 3 hours, mostly driving two browsers at once to test and fix a generated HTML/JS 3D game.

generation:

70.0 tokens/s

prompt processing (prefill):

1300.0 tokens/s

quant:

W4A16 (AutoRound)

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next at 70 t/s generation and 1.3k t/s prefill on 4x V620 (128GB total VRAM) at 128k+ context. Setup is a vLLM fork with AutoRound W4A16 quantization and MTP-2 speculative decoding, on an EPYC 7452 with 256GB DDR4. Prose generation runs at 60 t/s. Power draw is 700-900W during prefill and 500-600W during decode. User was initially disappointed with Qwen3.8 27B speeds but is satisfied with Flash-Next.

generation:

70.0 tokens/s

prompt processing (prefill):

1386.0 tokens/s

quant:

W4A16 (AutoRound)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 Flash-Next at 70.0 t/s generation and 1,386 t/s prompt processing on 4 Radeon Pro V620 GPUs at 32K context on the coding suite. Setup is vLLM with the Intel/Qwen3.8-Flash-Next-W4A16-AutoRound quant, using a custom RDNA vLLM fork. At 64K context the coding suite reached 72.5 t/s generation and 1,369 t/s prompt processing; at 128K it reached 68.3 t/s and 1,296 t/s. The regular suite measured 57.9, 59.5, and 56.1 t/s generation at 32K, 64K, and 128K respectively.

generation:

35.4 tokens/s

prompt processing (prefill):

355.4 tokens/s

quant:

Q6_K_XL (GGUF)

kv:

f16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Muse Glimmer 30B on AMD v620 GPUs, with Q6 on one GPU at 355.38 t/s prefill and 35.38 t/s generation. Setup is a tensor split across two GPUs: Q6 gives 472.27 t/s prefill and 36.32 t/s generation, Q8 gives 550.04 t/s prefill and 26.55 t/s generation. Speculative decoding uses a dflash draft model.

generation:

21.1 tokens/s

prompt processing (prefill):

276.2 tokens/s

quant:

IQ3_XXS

kv:

FP16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports DeepSeek-V4-Flash-0731 at 21.06 t/s continuous generation and 30.59 t/s sustained accepted generation on 4x AMD Radeon Pro V620 with 128 GB total. Setup is the unsloth IQ3_XXS quant with speculative decoding using a DSpark Q8 drafter on CPU RAM and 2 draft tokens. Prompt ingestion is 276.23 t/s at 32K and 379.01 t/s at 4K. The user calls the model at q3_xxs "fucking stupid" and lobotomized, and notes there is not enough VRAM for the drafter, which affects TG speeds.

generation:

30.0 tokens/s

prompt processing (prefill):

1000-1300 tokens/s

quant:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at Q8 running at 30 t/s with 1-1.3k t/s prefill on a Threadripper 3975WX with four Radeon Pro V620 cards. User measures electricity rather than speed: about 0.8-0.85 kW during inference and 0.15-0.2 kW idle, which at 0.32 dollars per kWh works out to roughly 55-60 dollars a month for six hours of inference a day. User calls 30 t/s pretty slow and reports 3.5k prefill and 70-80 t/s on Qwen3.6 35B instead.

generation:

14.3 tokens/s

prompt processing (prefill):

364.0 tokens/s

quant:

Q5_K_P (GGUF)

kv:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Qwen3.8-27B and Gemma4-26B-A4B on an AMD V620 using llama.cpp with ROCm and Vulkan. Qwen results are shown; Gemma results appear in a table but are not extracted as the primary model.