Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Search for a model to see its reported speeds across GPUs and Macs.
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
AMD RX 7900 XTX · hipfire
User reports Qwen3.8 27B averaging 95 t/s decode and 1500 t/s prefill on a single RX 7900 XTX using a hipfire fork. No quant, context length or KV cache settings are stated. The user calls the result a removal of AMD's governor and is enthusiastic about it.
AMD RX 7900 XTX · llama.cpp
User reports Qwen3.6-35B-A3B at 125.47 t/s generation on an RX 7900 XTX 24GB. Setup is llama.cpp (Vulkan backend, v0.7.0) via LM Studio with Q4_K_M GGUF and no speculative decoding. Three consecutive runs gave 125.47, 125.07 and 126.47 t/s generation; prompt processing was 1874.71 t/s over 427 tokens, 1288.99 t/s over 311 tokens and 770.07 t/s over 157 tokens.
AMD RX 7900 XTX · llama.cpp · 184,320 ctx
User reports Qwen3.8-27B at 69.63 t/s generation on an RX 7900 XTX 24GB, with prompt processing at 383.02 t/s. Setup is llama.cpp 2.61.0 (Vulkan/RADV) via LM Studio with Q4_K_M and q8_0 KV cache, 184320-token context, flash attention on, all layers on GPU and about 23.3 of 24 GiB VRAM in use. Speculative decoding was on (MTP draft plus ngram, up to 2 draft tokens), so decode is inflated versus a plain run; draft acceptance was 0.558-0.68. A second 512-token run reached 64.09 t/s decode and 239.97 t/s prompt. One request at a time, nothing offloaded to system RAM.
2× AMD RX 7900 XTX · Dspark
User reports DeepSeek V4.1 at 9 t/s decode average on an RX 7900 XTX 24GB plus W6800 32GB, with prefill around 150 t/s. Setup is Dspark with mostly IQ2 weights and Q6 n-grams, about 350GB on disk, streamed from a PCIe gen3 NVMe drive on a Ryzen 9 3950X with 96GB dual-channel DDR4 at 3000MHz. Decode varied widely: peaks of 19 t/s, usually around 13 t/s, dropping to 3 t/s when I/O was needed. User calls the model very smart but says it thinks a lot, so the slow average limits it to chatting.
AMD RX 7900 XTX · llama.cpp
User reports Qwen3.8 27B at 37.09 t/s generation and 877.84 t/s prompt processing on an AMD Radeon RX 7900 XTX under llama.cpp's Vulkan backend. Setup is llama.cpp (build 10438) with a UD-Q4_K_XL GGUF of 16.68 GiB, run with GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 because Resizable BAR is disabled in the BIOS; without that variable token generation is slow. At a 4096-token context depth the same run gives 36.33 t/s generation and 812.34 t/s prompt processing.
AMD RX 7900 XTX · llama.cpp · 106,496 ctx
User reports Qwen3.8-27B at 78.53 t/s generation and 523.33 t/s prompt processing on a Radeon RX 7900 XTX 24GB. Setup is llama.cpp 2.61.0 (Vulkan backend, LM Studio) with Q4_K_M GGUF and q8_0 KV cache at 106496 context, whole model on the single GPU (16.8GB of 24GB VRAM). Speculative decoding was on (MTP + ngram, spec-draft-n-max 2), so the generation figure is an effective speed with draft acceptance of 78/98. Four runs, one request at a time: generation ranged 61.68 to 78.53 t/s across runs. Runs 3 and 4 repeated a cached prompt (588 of 592 tokens), so their prompt speeds are startup time, not prefill. The user runs this model as a daily driver for coding and agent work.
AMD RX 7900 XTX · llama.cpp · 16,384 ctx
User reports Qwen3.6-27B at 41.70 t/s decode and 1075.8 t/s prompt processing on an RX 7900 XTX. Setup is llama.cpp (ROCm/HIP backend, LTO and WMMA flash attention on) with IQ4_XS weights and F16 KV cache at 16K context, batch 1024 and ubatch 256. The model is a 14.1 GiB MTP build; the user calls this the best overall configuration and notes LTO gives a 5-15% improvement.
AMD RX 7900 XTX · llama.cpp
User benchmarks a llama 7B Q4_0 model (6.74B params, 3.56 GiB) at 172.86 t/s generation and 3290.92 t/s prompt processing on a single AMD RX 7900 XTX. Setup is llama.cpp (build a81a56957) with the Vulkan backend, flash attention on, ngl 100 and n_batch 512, on Arch Linux with RADV Mesa 25.3.1. A dual-GPU run with 2x RX 7900 XTX was slower at 129.80 t/s generation and 2546.13 t/s prompt processing, which the user attributes to PCIe transfer overhead on small models.
AMD RX 7900 XTX · llama.cpp · 65,536 ctx
User reports Qwen3.8 27B at 31 t/s on a single RX 7900 XTX 24GB with 32GB DDR5 6000 system RAM. Setup is llama.cpp with Q4_K_M at 64K context. User feels the setup is not optimized for coding on large projects and asks for resources explaining how quantization, KV cache types, context size, batch size, GPU offloading and expert hits affect memory use, speed and quality.
User reports Qwen3.8 Flash-Next at around 50-60 t/s decode with 1300 t/s prefill on a dual RX 7900 XTX system. Setup is Strata with a Q3_K_S GGUF and Q8 KV cache, running on the rest of the system while a Qwen3.8 27B Q4_K_XL model with 131k context and Q8 KV sits on the second card behind the chipset at gen3 x4. The 27B gets around 40 t/s decode and 700-800 t/s prefill. An unused flag raises Flash-Next prefill to about 2300 t/s but makes inference unstable, and a layer split keeps decode the same while bringing prefill near 3000 t/s. User is experimenting with pi/hermes agent setups and is not satisfied with how the two models work together.
AMD RX 7900 XTX · llama.cpp
User compares Qwen3.8-27B at 47.9 tok/s on an RX 7900 XTX 24GB against Qwen3.8-Flash-Next at 36.8 tok/s on an MI50 32GB. Setup is llama.cpp with UD-Q4_K_XL and F16 KV cache for the 27B; Strata with IQ3_S and fp16 KV cache for Flash-Next. Both ran through an OpenAI-compatible API with temp 1.0, top_p 0.95, top_k 20. Speed figures cover AIME26 and LiveCodeBench generation (1.34M output tokens over 7.8 h for the 27B). The 27B scored 145/166 on the eval suite versus 151/166 for Flash-Next, a difference the user notes is not statistically significant (p = 0.18).
AMD RX 7900 XTX · llama.cpp · 384,000 ctx
User pruned GLM-5.3-Flash (321B MoE) from 288 to 144 routed experts per layer, selecting experts on agentic coding sessions, then quantized with an imatrix built on the same sessions. The IQ2_XXS build (45.3 GB) runs on an RX 7900 XTX (24 GB) plus 32 GB DDR4-2133 with an i7-7700K over PCIe 3.0, at roughly 16 tok/s generation and roughly 950 tok/s prompt processing. Setup is llama.cpp with IQ2_XXS GGUF weights and 384K context. The 2-bit version is experimental: it solved 39/48 attempts versus 48/48 for Q3_K and above, and when it fails it loops or keeps exploring until the turn limit. An anti-loop guard did not help (27/37 vs 30/37). Evaluation used 26 Python SWE tasks from ansible, openlibrary and qutebrowser, 2 attempts each, 200 turns, in a sealed clean room with no network or git history. The code-scored expert mask needed x1.21 turns versus the full model, against x2.21 for a frequency mask and x2.29 for the public multi-domain REAP 50% mask. IQ2_S wants 24 + 48 GB, Q3_K 24 + 64 GB, Q4_K_M 24 + 96 GB or 128 GB unified memory.
AMD RX 7900 XTX · 220,000 ctx
User asks for hardware upgrade advice for running local LLMs, currently using an RX 7900 XTX with a Ryzen 5600X and 64GB DDR4. They report running Qwen3.8 27B at Q4 with 128k to 220k context and Q8 KV cache, getting up to 50 t/s generation and around 750 t/s prompt ingestion. They are considering 2-4 MI50 16GB cards or a second 7900 XTX, and ask about motherboard recommendations.
AMD RX 7900 XTX · TabbyAPI · 196,608 ctx
User reports Qwen3.8-Flash-Next at 83 tok/s on code and 45-50 tok/s on prose on a single RX 7900 XTX 24GB with 60GB DDR5. Setup is TabbyAPI with EXL3 3.05bpw, 128K context and 8-bit KV cache, 166 of 512 experts per layer on the GPU and 12 CPU threads, MTP drafting with 2 tokens. A 28.7K-token code prompt prefills at 505 tok/s cold and decodes at 51-70 tok/s; without drafting the model runs 53 tok/s on code. The 4.05bpw variant reaches 68 tok/s on code. llama.cpp with IQ4_XS and --n-cpu-moe 36 is cited as a reference at 17.5 tok/s.
AMD RX 7900 XTX · ExLlamaV3 · 2,048 ctx
User reports Qwen3.8-27B at 157 tok/s decode on an RX 7900 XTX 24GB with DFlash2 speculative decoding at 2K context. Setup is a ROCm port of ExLlamaV3 with EXL3 3.5bpw weights, Q8 KV cache, and a DFlash2 5.0bpw draft model, served through TabbyAPI. Decode drops to 147 tok/s at 32K and 107 tok/s at 99K; MTP gives 108 tok/s at 2K. Prefill is ~1180-1200 tok/s at 32K. Through the TabbyAPI OpenAI endpoint, code reaches ~186 tok/s and prose ~78 tok/s. The same GPU with llama.cpp and an IQ3_XXS GGUF decoded at 40-53 tok/s.
2× AMD RX 7900 XTX · llama.cpp
User reports about 920 tokens/s prompt processing for Qwen3.8 Flash-Next MoE UD_Q3_XXL on two RX 7900 XTX cards, and about 1,600 tokens/s prompt processing for Qwen3.8 27B dense Q8_0 with tensor parallel. Setup is a custom llama.cpp RDNA3 fork with ROCm 7.14, adaptive MTP draft depth up to 3, Q8_0 KV cache, layer split, and MoE experts offloaded to system RAM with a 288-slot GPU-resident expert cache. The user notes the expert cache slows prompt processing to a crawl on their workload and they run with it disabled; with 288 experts they report a 95% hit rate and 45 t/s decode with MTP. The build is experimental and the figures are measured, not guaranteed.
4× AMD RX 7900 XTX · llama.cpp · 51,000 ctx
User reports Qwen3.8-27B Q8 at 77 t/s decode at 51K depth and 1,816 t/s prefill over a 51K prompt on four RX 7900 XTX cards after a series of ROCm and llama-server fixes. Setup is 4x RX 7900 XTX on PCIe 4.0 x16 in a Lenovo P620, Q8 weights, a locally patched llama.cpp with MTP speculative decoding. Before the fixes the same run gave 57 t/s decode at 51K and 1,454 t/s prefill; code decode went from 54-61 to 96-110 t/s. The patches are not yet published.
AMD RX 7900 XTX · llama.cpp · 262,144 ctx
User reports Qwen3.8-27B at 44.5 tok/s decode and 404 tok/s prefill on an RX 7900 XTX 24GB. Setup is llama.cpp (ROCm host build b11160 for gfx1100) with IQ3_S GGUF weights and q8_0/q5_1 KV cache, running the full 262,144-token window with MTP speculation on. Context gate recalled at 213,570 prompt tokens in 528.7 s. Paired MTP measurements for this launch are spec-off 11.2 tok/s versus 23.3 tok/s at 240K fill. The desktop shares about 1.2 GiB of the card, so a headless box may decode slightly faster.
AMD RX 7900 XTX · llama-server-fpx
User reports Qwen3.8-27B-ROCmFP4-FAST decoding at 12.8-13.2 t/s on an AMD RX 7900 XTX 24 GB, with later runs at 16.5-18.1 t/s, against an expectation of about 75 t/s. Setup is llama-server-fpx (ROCmFPx fork, Vulkan backend) with the 13.5 GiB GGUF, -ngl 999, flash attention on, and MTP speculative decoding (--spec-type draft-mtp, --spec-draft-n-max 4). Prompt processing is 42.7 t/s. MTP acceptance is 70-73%, later 79%. The machine is a Dell 5820 Tower with an Intel W-2125 and 128 GB RAM, 64 GB of it passed to a Docker LXC.
AMD RX 7900 XTX · llama.cpp · 262,144 ctx
User reports Qwen3.8-27B at 68.9 t/s on an RX 7900 XTX 24 GB at empty context with MTP n=3 speculative decoding, and 37.2 t/s without MTP. Setup is llama.cpp (ROCm, hip-kvmix) with IQ3_S GGUF and q8_0/q5_1 KV cache at 262,144-token context. At 240K fill with MTP the user reports 18.6-26.9 t/s (mean 23.3) and 380 t/s prefill, with 22,630 MiB peak VRAM.
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking