llamaperf

Qwen3 8B

on AMD RX 9070 XT 16GB

Tone: mixedAMD hardware
Nov 16, 2026
Throughput
68.4 t/s gen
Quant
B12
VRAM reported
16 GB

Summary

User reports Qwen3-8B at 68.4 tok/s with speculative decoding on an RX 9070 XT 16GB, versus 42.7 tok/s plain, using a custom engine that packs bf16 weights into a lossless 12-bit format (B12) with a Qwen3-0.6B draft model. Setup is the user's own engine on Windows with ROCm PyTorch, batch-invariant GEMV kernels and seeded Gumbel-max sampling; speculative output matched plain decoding on 11 of 11 prompts. On 50 longer prompts of 512 tokens the speculative run reached 73.0 tok/s with 50/50 matches. The user notes llama.cpp Q8_0 with a draft is about 1.4x faster (98.8 tok/s ROCm, 95.0 Vulkan) but lossy, matching plain output on only 7 of 11 prompts.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Qwen3 8B at 4-bit by hardware

Bars are medians on one scale, 0 to 130.6 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Models most run on the AMD RX 9070 XT 16GB

Bars are medians on one scale, 0 to 98.8 tokens/s. Press ▶ to watch text arrive at a row's speed.

Everything reported on the AMD RX 9070 XT 16GB →Every Qwen3 report →Estimate Qwen3 on this card →