Local model performance on your hardware

generation:

35.0 tokens/s

quant:

UD-Q5_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 35 t/s on an RX 7800 XT 16GB, with a Ryzen 5 7600X and 32GB DDR5. Setup is llama.cpp with UD-Q5_K_M (about 25GB), which spills past the 16GB card. User asks whether to swap to a used 7900 XTX 24GB or add a used 7600 XT 16GB, and requests real t/s numbers for either setup under Vulkan or ROCm.

generation:

110.0 tokens/s

quant:

UD-IQ3_XXS (GGUF)

kv:

turbo4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at ~110 t/s on an AMD RX 7800 XT 16GB with MTP and turbo4 KV cache at 32K context. Setup is a NJannasch llama.cpp mtp-turboquant fork on ROCm 6.4.0 (gfx1101) with UD-IQ3_XXS GGUF, turbo4 KV cache, --spec-type draft-mtp, and --reasoning-budget 512. MTP acceptance was 86% at 32K context; turbo4 KV saved ~470 MB vs f16 at 32K. The user warns that a silent CPU fallback can drop generation to 5-10 t/s and that gfx1032 cards crash with this fork.

generation:

30.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

q8_0/q5_1

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen 3.8 27B at ~30 t/s decode on an RX 7800 XT 16GB. Setup is llama.cpp with Vulkan, UD-IQ4_XS GGUF quant, q8_0/q5_1 KV cache, 100096 context, no MTP, single request, full GPU offload. User shares the setup because many believe Qwen 3.8 27B is infeasible on 16GB VRAM.