llamaperf

AMD RX 6700 XT

AMD · 12GB · 5 reports

Is this your card?

On the AMD RX 6700 XT, Gemma 4 12B at 4-bit, the model most run on it, generates a median of 34.6 tokens per second from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the AMD RX 6700 XT, each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

  • Gemma 4 12B4-bit · PP 653.9 t/s, one run
    34.6 t/s · one run
  • Qwen3 8B4-bit · PP 851.0 t/s, one run
    61.0 t/s · one run
  • 10.0 t/s · one run

Bars are medians on one scale, 0 to 61.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: llama.cpp 4 (reports)

Run models on your AMD RX 6700 XT? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the AMD RX 6700 XT

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 12 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: mixedAMD hardware
generation:
10.0 tokens/s
quant:
IQ3_XXS (GGUF)
coding

User reports Qwen3.8 27B at IQ3_XXS averaging around 10 t/s on an RX 6700 XT 12GB, with 32GB DDR4 3200 and a Ryzen 5 5600X. No engine or context length is stated for the run. The user says the MoE model they also run gets 35 t/s or more, and is weighing the DeepSeek harness against pi dev and opencode for finding a useful local workload.

Oct 28, 2026
Tone: mixedAMD hardware
generation:
30.0 tokens/s
quant:
Q4_K_XL (GGUF)
kv:
Q8_0
flash attention:
on
coding

User reports 30 t/s generation on an RX 6700 XT 12GB with a Qwen3.6 35B-A3B Tiel-Coder model at Q4_K_XL, using llama.cpp with the Vulkan backend on Windows 11. Setup uses 100,000 context, Q8_0 KV cache, flash attention, 28 MoE layers offloaded to CPU, 6 threads, and MTP speculative decoding with draft-n-max 3. User is asking for advice to reach 40 t/s without changing the quant, and notes 6 threads was best out of 4, 8, and 12.

Oct 11, 2026
Tone: positiveAMD hardware
generation:
61.0 tokens/s
prompt processing (prefill):
851.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
f16

User reports Qwen3-8B Q4_K_M at 61 t/s generation and 851 t/s prompt on an RX 6700 XT 12 GB. Setup is the bundled llama.cpp (0.85.5) with the AMD Flash Attention kernel, KV cache f16, measured with pp512 / tg128. The project's headline claim is that the same card goes from 0.6-2.6 t/s under stock llama.cpp to ~61 t/s with ToshLLM. The page also lists Qwen3.6-35B-A3B Q4_K_S at 475 t/s prompt / 29 t/s generation with --n-cpu-moe 24, and community rows including a 122B-A10B MoE across four W6800X GPUs at 226 t/s prompt / 26 t/s generation.

Oct 9, 2026

Gemma 4 12B

AMD RX 6700 XT · llama.cpp · 8,192 ctx

Tone: mixedAMD hardware
generation:
34.6 tokens/s
prompt processing (prefill):
653.9 tokens/s
quant:
IQ4_NL (GGUF)
kv:
q8_0
long-context

User benchmarks Gemma 4 12B (IQ4_NL, 6.24 GiB) on an AMD RX 6700 XT under llama.cpp, comparing the ROCm and Vulkan backends at 8192-token prefill and 512-token generation with q8_0 KV cache and flash-attention on. ROCm averages 653.9 t/s prefill and 34.60 t/s decode over 3 runs; Vulkan averages 354.4 t/s prefill and 40.92 t/s decode. ROCm is 84.5% faster on prefill but 15.4% slower on decode, giving a net wall-clock win of about 23% for a full 8192-prefill plus 512-generate cycle, with a crossover near 1760 prompt tokens. ROCm required two workarounds on gfx1031: building for gfx1030 with HSA_OVERRIDE_GFX_VERSION=10.3.0, and patching a flash-attention assert in fattn-common.cuh. A separate TOP_K sampler gap on ROCm is noted as under investigation.

Oct 6, 2026

Qwen3.8 27B

2× AMD RX 6700 XT · llama.cpp · 144,000 ctx

Tone: positiveAMD hardware
generation:
18.0 tokens/s
prompt processing (prefill):
150.0 tokens/s
quant:
UD-IQ3_XXS (GGUF)
agenticlong-context

User reports Qwen3.8 27B at 18 t/s generation and 150 t/s prompt processing on a 2007 Dell Precision T5400 with dual RX 6700 XT / RX 6700 (22GB total VRAM). Setup is llama.cpp with UD-IQ3_XXS quant at 144k context, using MTP q4_0 speculative decoding, on 24GB DDR2 and dual Xeon X5460. User compares five systems and argues older dual-Xeon dual-GPU setups beat a 2025 HP Omen with RTX 5070 in context length and speed, concluding DDR5 is not worth the cost for agentic tasks.

Sep 28, 2026

Get a weekly email of new AMD RX 6700 XT reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3.8 27B
AMD RX 6700 XT
IQ3_XXS
Engine not reported
Not reported10.0 tokens/s
Qwen3.6 35B (3B active) Tiel-Coder
AMD RX 6700 XT
Q4_K_XL
llama.cpp
100,00030.0 tokens/s
Qwen3 8B
AMD RX 6700 XT
Q4_K_M
llama.cpp
Not reported61.0 tokens/s
Gemma 4 12B
AMD RX 6700 XT
IQ4_NL
llama.cpp
8,19234.6 tokens/s
Qwen3.8 27B
2× AMD RX 6700 XT
UD-IQ3_XXS
llama.cpp
144,00018.0 tokens/s

Frequently asked

How fast is the AMD RX 6700 XT for local LLMs?

On the AMD RX 6700 XT, Gemma 4 12B at 4-bit, the model most run on it, generates a median of 34.6 tokens per second from one plain run (community reports, as of 22 Nov 2026). Also reported: Qwen3 8B at 4-bit (61.0 t/s, one plain run); Qwen3.8 27B at 3-bit (10.0 t/s, one plain run).

What is the biggest model reported running on the AMD RX 6700 XT?

The largest model with a plain generation run on the AMD RX 6700 XT is Qwen3.8 27B at 3-bit (10.0 t/s, one plain run), as of 22 Nov 2026. Plain means one device, one request, no speculative decoding, so a run split across several cards does not count.