Unknown family
on AMD RX 7900 XT · Strata · 196,608 ctx
Use cases
coding
Summary
User reports a median decode of 53.3 tok/s over 95 generations on an RX 7900 XT, running an Unsloth Dynamic UD-IQ1_M GGUF on the Strata RDNA3 fork (engine build 0.1.59).
Setup uses int8 KV cache with 65536 of 196608 cells resident, an expert cache in VRAM with a 512 MiB reserve, experts in pinned RAM, MTP speculation with 4 draft tokens, and a 16384-token prefill chunk. The 68 GB model runs with 712 MiB of VRAM to spare on a 39 GB machine.
Cold prompt reads were a median 2032 tok/s at 5K-20K, 2215 at 20K-50K and 1676 at 50K-100K, with the first big prompt of a run dropping to 950-1100 tok/s. A separate 192K-context run had a decode median of 54.6. The user also ported an SSD conversation cache, which currently loses to the 192K context on RAM.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Models most run on the AMD RX 7900 XT
- 108.9 t/s · one run
- Qwen2.5 7B4-bit115.0 t/s · one run
- Qwen3.8 27B4-bit32.0 t/s · one run
- 4.6 t/s · one run
Bars are medians on one scale, 0 to 115.0 tokens/s. Press ▶ to watch text arrive at a row's speed.