llamaperf

Qwen3.8 Flash

on 2× AMD RX 7900 XTX · llama.cpp · 262,144 ctx

AMD hardware
Sep 28, 2026
Throughput
23.7 t/s gen · 429.7 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
IU4 (GGUF)
KV cache
F16
System RAM
128 GB

Use cases

agenticlong-contextcoding

Summary

User reports Qwen3.8 Flash at 429.68 t/s prefill and 23.74 t/s decode on an RX 7900 XTX 24GB plus Strix Halo 128GB over OCuLink. Setup is a custom llama.cpp runtime with IU4 GGUF weights, F16 KV cache, MTP depth 3, batch/microbatch 1536, eight CPU threads and one inference slot, targeting 262,144 tokens of context. The full-context request processed 261,888 prompt tokens in 609.49 seconds and generated 128 tokens in 5.35 seconds. A HermesAgent-20 agent workload with prompt caching on reports 590.24 t/s prefill and 50.00 t/s decode across 120 requests, with a decode range of 42.07-56.50 t/s for responses of at least 100 tokens. The user notes the approximately 50 t/s decode is an agent-workload result, not a filled-256K measurement.