AMD RX 6800 XT 16GB

generation:

18.5-19.4 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen3.8-27B at 18.5-19.4 tok/s decode on two RX 6800 XT/6800 16GB cards with layer split. Setup is llama.cpp via Ollama 0.33.2 with Q4_K_M, 65536 context, MTP speculative decoding (draft_num_predict 2). Prefill is ~230 tok/s on a fresh 15k-token prompt; cached runs showed 3,600-13,000 tok/s. Forced flash-attn dropped decode to 16.4 tok/s. User asks whether ~58% of theoretical bandwidth (297 GB/s of 512 GB/s) is the expected ceiling for hybrid linear-attention models on ROCm, and notes Vulkan crashes mid-decode with DeviceLostError.

generation:

45.0 tokens/s

prompt processing (prefill):

450.0 tokens/s

quant:

Q6_K (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Qwen3.8 27B at 45 t/s generation and 450 t/s prompt processing on dual RX 6800 XT GPUs under Linux with ROCm. Setup is llama.cpp with Q6_K weights and Q8_0 KV cache, using MTP speculative decoding and tensor parallelism across two gfx1030 cards. User compares Windows 11 Vulkan results: 35 t/s TG and 180 t/s PP with MTP, 20 t/s TG and 360 t/s PP without MTP, and 15 t/s TG and 350 t/s PP with tensor parallelism and no MTP. User notes a Windows Vulkan bug that halves prompt processing and asks about performance ceilings and other engines.