Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.
- generation:
- 15-18 tokens/s
- quant:
- Q4 (GGUF)
User reports running Qwen3.8 27B Q4 via llama.cpp RPC across two machines, an RX 9060 XT 16GB and an RTX 5060 Ti 16GB, at 15-18 t/s with a 32k context.
Setup is llama.cpp RPC with Q4 weights and a 32k context, split over the two 16GB cards.
The user asks whether Qwen3.8 Flash-Next can be run by adding CPU-only RPC workers on the same machines to offload to their 32GB and 64GB of system RAM.
- generation:
- 20-29 tokens/s
- quant:
- IQ3_S (GGUF)
- kv:
- q5_0/q4_1
codingagentictool-usevision
User reports running Swift-1.5-Qwen3.8-27B at IQ3_S on an RX 9060 XT 16GB, with eval in the mid 20s t/s and prompt processing between 550 and 231 t/s.
Setup is llama.cpp (llama-server) with MTP draft speculation, q5_0 K and q4_1 V caches, 98304 context, all layers on the GPU and the mmproj kept on the CPU. Draft acceptance was around 0.6-0.8.
A web-search agent request took 2m 10s end to end; the user has seen mid-40s eval at other settings. They ask whether to shrink context, add draft KV quantisation, and how to handle vision.
- generation:
- 45.5 tokens/s
- prompt processing (prefill):
- 161.9 tokens/s
- quant:
- Q2_0 (GGUF)
- kv:
- int8
chat
User reports Qwen3.8-Flash-Next Q2_0 at 45.5 t/s decode and 161.9 t/s prefill (pp4k) on an RX 9060 XT 16GB with 32GB DDR4, but says prefill is usually under 150 t/s and asks how to fix it.
Setup is Strata 0.1.52 (HIP backend, Windows) with a 64K context and int8 KV cache; the 37.6 GB GGUF exceeds the card's VRAM, so weights run partly from system RAM.
Strata's tuner ran 501.5 s and confirmed the defaults; two tuning arms were skipped for lack of free memory (25.6 GiB free, 28.4 GiB needed).
- generation:
- 40-55 tokens/s
- quant:
- IQ3_S (GGUF)
- kv:
- Q8
coding
User reports running Qwen 3.8 Flash-Next IQ3_S and Swift 1.5 IQ3_XXS on an RX 9060 XT 16GB with a 7800X3D and 64GB RAM.
Setup is Strata, with Flash-Next at a 130K context window and Q8 KV cache, and Swift 1.5 at 262K context and Q4 KV cache.
Token generation stayed between 40-55 tk/s across the whole context window for both models. Prompt processing was around 800 tk/s on resumed sessions and around 900 tk/s during compaction. Neither model could build the requested Figma plugin in one shot, but the user is happy with performance and plans to stay on the 9060 XT.
- generation:
- 70.7 tokens/s
- prompt processing (prefill):
- 1326.0 tokens/s
- quant:
- UD-Q3_K_M (GGUF)
- kv:
- FP16
User reports Qwen3.6-35B-A3B at 70.7 tok/s generation on a 31,296-token prompt on an RX 9060 XT 16GB with a Ryzen 5 5600X and 32GB RAM on Windows 11.
Setup is the PolyStrata-HIP fork of the strata-poly GGUF engine with Unsloth's UD-Q3_K_M quant, FP16 KV cache and MTP speculation enabled; experts not resident in VRAM are computed on the CPU from system RAM.
Prefill on that run was 1,326 tok/s. Shorter warm prompts reached 93.4 tok/s at 2,068 tokens and 85.8 tok/s at 8,235 tokens, while a 45,355-token prompt gave 61.9 tok/s. With Q8_0 KV and 262,144 tokens of allocated capacity, a 400-token response ran at 33.8 tok/s capped at 2,000 resident experts versus 47.6 tok/s uncapped with 5,097 resident. Streaming KV through pinned RAM at roughly 261K tokens dropped generation to 3.64 tok/s.
- generation:
- 42.7 tokens/s
- prompt processing (prefill):
- 658.0 tokens/s
- quant:
- IQ4_XS (GGUF)
- kv:
- Q4/Q8
- mtp (multi-token prediction):
- on
User reports Qwen 3.8 27B at 42.7 t/s generation and 658 t/s prefill at 20k context on a single RX 9060 XT 16GB.
Setup is a fork of llama.cpp (achyudh/llama.cpp) with an unsloth IQ4_XS GGUF, Q4 KV values with Q8 keys, MTP plus n-gram prediction, and KV cache streaming; 24GB of DDR5 system RAM, 16GB of it used for this model.
At 200k context the user reports 35 t/s generation and 300 t/s prefill, with a 262k context target. The user notes the build needed PCIe atomics fixes.
- generation:
- 66.0 tokens/s
- quant:
- UD-Q4_K_XL (GGUF)
- kv:
- q8_0
User reports Qwen3.6-35B-A3B at 66.04 tokens/s on an AMD Radeon RX 9060 XT 16 GB with 32 GB of system RAM.
Setup is llama.cpp (Vulkan backend) with UD-Q4_K_XL GGUF weights, q8_0 KV cache, 40k context, and MTP speculative decoding drafting up to 3 tokens per step. The 35B MoE model does not fit in 16 GB of VRAM, so the expert weights of the first 20 layers run on the CPU.
Run on a Ryzen 7 5800X3D desktop under SteamOS, single slot, with flash attention and prefix cache reuse enabled.
- generation:
- 30.0 tokens/s
- quant:
- IQ3_S (GGUF)
User asks whether Qwen3.8 Flash-Next can run on an RX 9060 XT 16GB with 32GB DDR5-6000.
They currently run Swift-1.5-Qwen3.8 at IQ3_S at 30 t/s on the same machine.
- generation:
- 50.0 tokens/s
- prompt processing (prefill):
- 850.0 tokens/s
- quant:
- IQ3_S (GGUF)
User reports 850 t/s prefill and 50 t/s generation with MTP at temperature 0 on an AMD RX 9060 XT 16GB, running Qwen3.8-27B-GSQ-RCO IQ3_S in a custom inference engine.
MTP is used for the generation figure; the user notes MTP with temperature above 0 is not developed yet.
User asks for ideas to systematically test the engine, having already run KL divergence against BF12 on CPU, bit correctness checks, needle-in-a-haystack at 25/50/75/90% key positions with distractors (also against llama.cpp), and HumanEval (92/93 passed so far).
- generation:
- 33.5 tokens/s
- prompt processing (prefill):
- 1210.3 tokens/s
- quant:
- Q4_K_M (GGUF)
User benchmarks Qwen 2.5-Coder 14B Instruct Q4_K_M at 33.51 t/s generation and 1210.29 t/s prompt processing on an AMD RX 9060 XT 16GB via Vulkan.
Setup is llama.cpp build c4ae9a88f8 with -ngl 99 and -fa 1; the same model on ROCm reaches 30.79 t/s generation and 1211.32 t/s prompt processing.
A Qwen 3.5 9B Q4_K_M run reaches 50.15 t/s generation and 1904.16 t/s prompt processing on Vulkan, and a 14B quant sweep gives about 32, 28 and 20 tok/s for Q4_K_M, Q5_K_M and Q8_0.
- generation:
- 36.0 tokens/s
- prompt processing (prefill):
- 950.0 tokens/s
- quant:
- Q6_K (GGUF)
- kv:
- Q8
codingagentic
User reports Ornith-1.5-9B at about 36 t/s generation and about 950 t/s prompt eval on an AMD RX 9060 XT 16GB with llama.cpp.
Throughput drops to about 25 t/s generation and about 500 t/s prompt eval during long tasks.
The run lasted about 3.5 hours on a coding agent task. The user contrasts this with Qwen 3.8 27B, which was frustratingly slow on the same hardware.