llamaperf

AMD RX 6700 XT reports

5 reports. These are individual setups, not a controlled benchmark. AMD RX 6700 XT speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: mixedAMD hardware
generation:
10.0 tokens/s
quant:
IQ3_XXS (GGUF)
coding

User reports Qwen3.8 27B at IQ3_XXS averaging around 10 t/s on an RX 6700 XT 12GB, with 32GB DDR4 3200 and a Ryzen 5 5600X. No engine or context length is stated for the run. The user says the MoE model they also run gets 35 t/s or more, and is weighing the DeepSeek harness against pi dev and opencode for finding a useful local workload.

Oct 28, 2026
Tone: mixedAMD hardware
generation:
30.0 tokens/s
quant:
Q4_K_XL (GGUF)
kv:
Q8_0
flash attention:
on
coding

User reports 30 t/s generation on an RX 6700 XT 12GB with a Qwen3.6 35B-A3B Tiel-Coder model at Q4_K_XL, using llama.cpp with the Vulkan backend on Windows 11. Setup uses 100,000 context, Q8_0 KV cache, flash attention, 28 MoE layers offloaded to CPU, 6 threads, and MTP speculative decoding with draft-n-max 3. User is asking for advice to reach 40 t/s without changing the quant, and notes 6 threads was best out of 4, 8, and 12.

Oct 11, 2026
Tone: positiveAMD hardware
generation:
61.0 tokens/s
prompt processing (prefill):
851.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
f16

User reports Qwen3-8B Q4_K_M at 61 t/s generation and 851 t/s prompt on an RX 6700 XT 12 GB. Setup is the bundled llama.cpp (0.85.5) with the AMD Flash Attention kernel, KV cache f16, measured with pp512 / tg128. The project's headline claim is that the same card goes from 0.6-2.6 t/s under stock llama.cpp to ~61 t/s with ToshLLM. The page also lists Qwen3.6-35B-A3B Q4_K_S at 475 t/s prompt / 29 t/s generation with --n-cpu-moe 24, and community rows including a 122B-A10B MoE across four W6800X GPUs at 226 t/s prompt / 26 t/s generation.

Oct 9, 2026

Gemma 4 12B

AMD RX 6700 XT · llama.cpp · 8,192 ctx

Tone: mixedAMD hardware
generation:
34.6 tokens/s
prompt processing (prefill):
653.9 tokens/s
quant:
IQ4_NL (GGUF)
kv:
q8_0
long-context

User benchmarks Gemma 4 12B (IQ4_NL, 6.24 GiB) on an AMD RX 6700 XT under llama.cpp, comparing the ROCm and Vulkan backends at 8192-token prefill and 512-token generation with q8_0 KV cache and flash-attention on. ROCm averages 653.9 t/s prefill and 34.60 t/s decode over 3 runs; Vulkan averages 354.4 t/s prefill and 40.92 t/s decode. ROCm is 84.5% faster on prefill but 15.4% slower on decode, giving a net wall-clock win of about 23% for a full 8192-prefill plus 512-generate cycle, with a crossover near 1760 prompt tokens. ROCm required two workarounds on gfx1031: building for gfx1030 with HSA_OVERRIDE_GFX_VERSION=10.3.0, and patching a flash-attention assert in fattn-common.cuh. A separate TOP_K sampler gap on ROCm is noted as under investigation.

Oct 6, 2026

Qwen3.8 27B

2× AMD RX 6700 XT · llama.cpp · 144,000 ctx

Tone: positiveAMD hardware
generation:
18.0 tokens/s
prompt processing (prefill):
150.0 tokens/s
quant:
UD-IQ3_XXS (GGUF)
agenticlong-context

User reports Qwen3.8 27B at 18 t/s generation and 150 t/s prompt processing on a 2007 Dell Precision T5400 with dual RX 6700 XT / RX 6700 (22GB total VRAM). Setup is llama.cpp with UD-IQ3_XXS quant at 144k context, using MTP q4_0 speculative decoding, on 24GB DDR2 and dual Xeon X5460. User compares five systems and argues older dual-Xeon dual-GPU setups beat a 2025 HP Omen with RTX 5070 in context length and speed, concluding DDR5 is not worth the cost for agentic tasks.

Sep 28, 2026