RX 7900 XTX

reported speed:

30.1 tokens/s generation · 422.3 tokens/s prompt processing

quant:

Q4_K_XL (GGUF)

kv:

Q4_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User benchmarks Gemma 4 31B on an RX 7900 XTX, reporting 30.13 t/s generation and 422.27 t/s prompt processing with ROCm. Setup is llama.cpp on Windows 10 with the UD-Q4_K_XL GGUF, flash attention, 32K context, all layers on GPU, and a Q4_0 KV cache. Vulkan single-GPU gave 24.81 t/s generation and 117.29 t/s prompt, while Vulkan multi-GPU with an added RTX 3080 gave 26.18 t/s generation and 74.69 t/s prompt. User notes ROCm started at maximum generation speed and declined with context, whereas Vulkan briefly sped up before declining.

reported speed:

24.0 tokens/s generation · 920.0 tokens/s prompt processing

quant:

Q3_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Qwen3.8 Next Q3_K_XL at 920 t/s prefill on 8192 tokens, 24/27 t/s prose and 40+ t/s code with MTP on two 7900 XTX cards. Setup is a custom llama.cpp build optimized for the 7900 XTX with tensor parallel across two cards. User also benchmarks Qwen3.8 27B Q8_0 at 1600 t/s prefill on 8192 tokens, 60/65 t/s prose and 100+ t/s code, and Qwen3.6 27B Q4_K_M on a single card at 1020 t/s prefill on 8192 tokens, 58/60 t/s prose and 75/80 t/s code.

reported speed:

100.0 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports DiffusionGemma 26B-A4B-it at about 100 t/s generation on 4x RX 7900 XTX, or 45-60 t/s total including prompt processing. Setup is vLLM on the dgemma branch with the diffusion sampler entropy_bound, a 152,671-token KV cache, and max concurrency 1.16x at 131k context.

reported speed:

12.0 tokens/s generation · 141.4 tokens/s prompt processing

quant:

UD-IQ3_XXS

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports prompt processing in the high 140s t/s, dropping to the mid 80s at 60k context, with generation at ~11.9 t/s. Setup uses the ROCm 7.2.4 backend on 1x 7900 XTX 24GB plus 3x MI60 32GB. One MI60 failed.

reported speed:

72.1 tokens/s generation

quant:

Q4_K_M

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User compares Qwen3.8-27B against Qwen3.6-27B on a fact-extraction task, where Qwen3.8 scores 0.7030 F1 against Qwen3.6's 0.7177, a statistical tie. Decode throughput falls from 85.6 to 72.1 t/s, about 16%, under the closest saved configs, though the runs used different llama.cpp builds. Qwen3.8 produces shorter answers, lowering end-to-end latency. The user expected larger gains from public benchmarks, and sees small gains on task-specific tests but massive gains only on benchmarks the model was trained on.

reported speed:

25.0 tokens/s generation

quant:

Q4_K_M (gguf)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Qwen3.8 27B Q4_K_M at 25-35 t/s on an RX 7900 XTX at 80k+ context, dropping to 20-21 t/s at 150k-180k context. Setup is llama.cpp with MTP enabled and a q8_0 KV cache with a q5_1 K cache. User expresses frustration over the speed and power consumption.

reported speed:

27.0 tokens/s generation · 573.0 tokens/s prompt processing

quant:

Q6_K_XL (gguf)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports llama.cpp benchmarks on a dual-GPU setup of an RTX 4090 laptop and an RX 7900 XTX over TB4, reaching full 262k context with +70% generation and +40% prefill. Setup uses the CUDA backend for the NVIDIA card and Vulkan for the AMD card, with a q8_0 KV cache to enable 262k context at some cost in speed. MTP speculative decoding gave +50-70% generation but halved prefill, and the user filed a bug. A Q6_K_XL quant over Q8_0 gave +9% generation and more context on CUDA, and rebuilding to a newer llama.cpp gave +13% generation for free.

reported speed:

14.4 tokens/s generation · 448.0 tokens/s prompt processing

quant:

UD-Q4_K_XL (gguf)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a custom llama.cpp Vulkan backend patch for Qwen3.8-Flash-Next improves prompt processing from 346 to 448 t/s and generation from 13.3 to 14.4 t/s at 2.7k context. At 131k context, prompt processing improves from 195 to 263 t/s and generation from 9.1 to 10.4 t/s. The patch fixes a top-k fallback to CPU in sparse attention. Greedy outputs are identical, the needle is found at 131k, and images are read correctly.

reported speed:

26.0 tokens/s generation

quant:

IQ4_XS (gguf)

kv:

Q8

mtp (multi-token prediction):

on

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports 22-30 t/s generation, averaging about 26 t/s, at 140k context. Setup is llama-server via llama-swap with a UD quant, an MTP draft with spec-draft-p-min 0.70, and a q8_0 KV cache, using about 22.1 GB of VRAM. The user runs it for coding with opencode.

reported speed:

11.0 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports 11 t/s on Qwen 3.8 with 2x RX 7900 XTX and 128 GB DDR4. User compares the result to 3090s and 9700s.

reported speed:

34.4 tokens/s generation · 215.8 tokens/s prompt processing

quant:

Q4_K_M (gguf)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User compares Ollama/ROCm and llama.cpp/Vulkan on an RX 7900 XTX. Ollama reaches 215.8 t/s prompt processing and 34.4 t/s generation at 64K context. llama.cpp/Vulkan reaches 192.0 t/s prompt and 35.8 t/s generation at 64K context, and 230.5 t/s prompt and 36.0 t/s generation at 8K context. The user notes Vulkan is only about 4% faster for generation, while Ollama is better at prompt processing at 64K. The user wonders whether higher reported speeds of 60-100 t/s come from MTP/speculative decoding or other optimizations.

User benchmarks Gemma 4 QAT against regular quants on an AMD 7900 XTX. No token/s figures are reported; wall clock times show speedups, including 45% faster and an 83% throughput increase for the 12B QAT. Models tested are 12B, 26B, 31B and E4B. The user reports quality as identical.

reported speed:

32.0 tokens/s prompt processing

quant:

Q6 (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User is considering adding a second 7900 XTX for 48 GB VRAM to run larger models. Currently runs Qwen 27B Q6 dense with 32K context at 32 t/s prompt processing. Main use case is coding via opencode.

reported speed:

40.0 tokens/s generation

quant:

Q4_K_M (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Gemma 4 E4B at 35-45 t/s on an RX 7900 XTX. Setup is Ollama with the Q4_K_M quant. User calls it the best consumer AMD option.

reported speed:

58.0 tokens/s generation · 83.0 tokens/s prompt processing

quant:

FP16 (safetensors)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports Gemma 4 E4B at 57.96 gen tok/s and 82.96 prompt tok/s on an RX 7900 XTX. Setup is vLLM with ROCm on the default path. Source is flexinfer.ai.