AMD RX 7800 XT 16GB

generation:

75.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Tiel-Coder-35B-A3B at around 75 tok/s decode on short prompts on an RX 7800 XT 16GB, dropping to about 49 tok/s at 100K context. Setup is shura.cpp, a llama.cpp fork with an automatic VRAM governor that keeps 1.5 GiB of VRAM free for the desktop, on a Ryzen 5 5600 with 32GB DDR4-3200 under Fedora and ROCm 7.2.4. Experts are cached on the GPU with the rest in system RAM. Prefill improved 16.3% at 28K context and 7.9% at 100K over the previous build, with short-context decode basically unchanged. KL divergence was 0.0346 against a 0.0343 self-comparison baseline. A 40-minute stress test of 384 requests mixing 28-34K prompts had no failed requests. Only AMD gfx1101 with ROCm was benchmarked; CUDA builds compile but are untested.

generation:

40-45 tokens/s

quant:

GSQ RCO

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User asks whether any engines are optimized for consumer AMD hardware, noting most specialized engines target Nvidia. They report running Swift 1.5 GSQ RCO at about 40-45 TPS on an RX 7800 XT, with no engine named. The figure is given only as a range, and the user says they are happy with the setup and wonders how much more can be squeezed out of the hardware.

generation:

40.0 tokens/s

quant:

4bit

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 4bit running at around 40 t/s on an RX 7800 XT 16GB with 32GB RAM, at a max context of 120k. User is considering upgrading to a used RX 7900 XTX for €900 to reach 250k context and replace AI subscriptions, but finds the current quality not great. User is not interested in Qwen3.8 Flash-Next.

generation:

35.0 tokens/s

quant:

UD-Q5_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at 35 t/s on an RX 7800 XT 16GB, with a Ryzen 5 7600X and 32GB DDR5. Setup is llama.cpp with UD-Q5_K_M (about 25GB), which spills past the 16GB card. User asks whether to swap to a used 7900 XTX 24GB or add a used 7600 XT 16GB, and requests real t/s numbers for either setup under Vulkan or ROCm.

generation:

64.9 tokens/s

prompt processing (prefill):

985.8 tokens/s

quant:

IQ3_XXS (GGUF)

kv:

int8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8-Flash-Next at 64.9 tok/s aggregate decode and 985.8 tok/s prefill on an RX 7800 XT 16GB. Setup is Strata 0.1.42 with IQ3_XXS and int8 KV cache, 39.97 GiB of expert weights in host RAM and 5,048 experts (8.19 GiB) cached in VRAM. MTP was enabled with --spec 4. The 64.9 tok/s is aggregate decode over 54 requests; median per-request was 66.4 tok/s with a 56.1-82.8 tok/s range. Other fresh-prompt prefill readings were 912.9 tok/s at 6,731 tokens, 1,004.6 tok/s at 76,599 tokens, 791.0 tok/s at 116,392 tokens and 509.8 tok/s at 38,790 tokens. Across all 61 tail-skip requests aggregate decode was 52.7 tok/s. An informal comparison gave 50.8 tok/s without route tail skip and 56.7 tok/s with it.

generation:

110.0 tokens/s

quant:

UD-IQ3_XXS (GGUF)

kv:

turbo4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.6-35B-A3B at ~110 t/s on an AMD RX 7800 XT 16GB with MTP and turbo4 KV cache at 32K context. Setup is a NJannasch llama.cpp mtp-turboquant fork on ROCm 6.4.0 (gfx1101) with UD-IQ3_XXS GGUF, turbo4 KV cache, --spec-type draft-mtp, and --reasoning-budget 512. MTP acceptance was 86% at 32K context; turbo4 KV saved ~470 MB vs f16 at 32K. The user warns that a silent CPU fallback can drop generation to 5-10 t/s and that gfx1032 cards crash with this fork.

generation:

1.5 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Llama 3.1 8B at 1.5 t/s on an AMD RX 7800 XT under Ollama, compared with 65 t/s in LM Studio on the same machine. Setup is Ollama 0.4.7 on Windows with the Q4_K_M quant, Adrenalin Edition 24.10.1 drivers. The user asks whether the gap is their setup or an Ollama limitation.

generation:

30.0 tokens/s

quant:

UD-IQ4_XS (GGUF)

kv:

q8_0/q5_1

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen 3.8 27B at ~30 t/s decode on an RX 7800 XT 16GB. Setup is llama.cpp with Vulkan, UD-IQ4_XS GGUF quant, q8_0/q5_1 KV cache, 100096 context, no MTP, single request, full GPU offload. User shares the setup because many believe Qwen 3.8 27B is infeasible on 16GB VRAM.