AMD RX 7900 XTX reports

generation:

22-30 tokens/s

quant:

IQ4_XS (GGUF)

kv:

Q8

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 22-30 t/s generation at 140k context. Setup is llama-server via llama-swap with a UD quant, an MTP draft with spec-draft-p-min 0.70, and a q8_0 KV cache, using about 22.1 GB of VRAM. The user runs it for coding with opencode.

generation:

11.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports 11 t/s on Qwen 3.8 with 2x RX 7900 XTX and 128 GB DDR4. User compares the result to 3090s and 9700s.

generation:

34.4 tokens/s

prompt processing (prefill):

215.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User compares Ollama/ROCm and llama.cpp/Vulkan on an RX 7900 XTX. Ollama reaches 215.8 t/s prompt processing and 34.4 t/s generation at 64K context. llama.cpp/Vulkan reaches 192.0 t/s prompt and 35.8 t/s generation at 64K context, and 230.5 t/s prompt and 36.0 t/s generation at 8K context. The user notes Vulkan is only about 4% faster for generation, while Ollama is better at prompt processing at 64K. The user wonders whether higher reported speeds of 60-100 t/s come from MTP/speculative decoding or other optimizations.

User benchmarks Gemma 4 QAT against regular quants on an AMD 7900 XTX. No token/s figures are reported; wall clock times show speedups, including 45% faster and an 83% throughput increase for the 12B QAT. Models tested are 12B, 26B, 31B and E4B. The user reports quality as identical.

prompt processing (prefill):

32.0 tokens/s

quant:

Q6 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User is considering adding a second 7900 XTX for 48 GB VRAM to run larger models. Currently runs Qwen 27B Q6 dense with 32K context at 32 t/s prompt processing. Main use case is coding via opencode.

generation:

35-45 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 35-45 t/s on an RX 7900 XTX. Setup is Ollama with the Q4_K_M quant. User calls it the best consumer AMD option.

generation:

58.0 tokens/s

prompt processing (prefill):

83.0 tokens/s

quant:

FP16 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Gemma 4 E4B at 57.96 gen tok/s and 82.96 prompt tok/s on an RX 7900 XTX. Setup is vLLM with ROCm on the default path. Source is flexinfer.ai.