- generation:
- 133.0 tokens/s
- quant:
- Q8_0 (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks Qwen3-8B Q8_0 with DSpark speculative decoding in llama.cpp on an RX 7900 GRE 16GB, reaching 133 t/s on ROCm versus 59 t/s without speculation, and 115 t/s versus 62 t/s on Vulkan.
Setup is llama.cpp with Q8_0 weights, greedy decoding, one user and 32 prompts per setting.
DFlash gave 1.5-1.9x, a 0.6B draft model about 1.3x, and n-gram lookup did nothing. The best number of guessed tokens is 3 on Vulkan and 7 on ROCm, and prompt processing is about 8% slower with DSpark loaded. Only 5-6 of 16 answers matched normal decoding word for word.
- generation:
- 51.9 tokens/s
- prompt processing (prefill):
- 322.0 tokens/s
- quant:
- Q4_K_M (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a 36% improvement in generation speed with a dual GPU setup of an RX 7900 GRE 16GB and an RX 480 8GB.
The user tested medgemma-27b-it-UD-Q6_K_XL, Qwen3.8-27B-Q6_K, Qwen3.8-27B-OBLITERATED-Q5_K_M, and Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M, with and without Flash Attention.
- generation:
- 11.8 tokens/s
- prompt processing (prefill):
- 87.5 tokens/s
- quant:
- Q4_K_M (GGUF)
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.8-27B, identified as Qwen3.5 in llama.cpp, using llama.cpp RPC across multiple GPUs.
Setup is an RX 7900 GRE 16GB as the main host, with RPC combinations including RX 7900 GRE + GTX 1080Ti and GTX 1080Ti + two P102-100, in Q4_K_M and Q6_K quants.
Results vary by config; the best generation is about 12.53 t/s with Q6_K on RX 7900 GRE + GTX 1080Ti.