AMD · 32GB · 6 reports
Use the calculator to check which models fit in 32 GB of VRAM. The reports below include different GPU counts and memory setups; a model listed here may need extra cards or CPU offloading.
How to compare these reports →3× AMD MI50 32GB · llama.cpp · 130,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports 15 t/s generation and 100-200 t/s prompt processing at 130k context. Setup is ROCm with flash-attn, split-mode layer and fit on. The user notes Vulkan gives similar speed.
AMD MI50 32GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User benchmarks the gfx906-llama-cpp fork on an MI50 32GB, reaching 410 t/s prefill at PP16384 against 332.5 t/s upstream, 264 t/s on a 120k deep fill against 231.4 t/s upstream, and 15.1 t/s TG against 13.6 t/s upstream. Outputs are bit-identical to upstream. The user attributes the gains to exploring existing llama.cpp PRs.
3× AMD MI50 32GB · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at 15-16 t/s generation and 105-110 t/s prompt processing on 3x AMD MI50 32GB GPUs, 96GB of VRAM in total. Setup is llama-server (llama.cpp) with the UD-IQ2_M quant. The 90.9GB model fits fully in VRAM. The user tested it with a Rubik's cube HTML coding task and is impressed it runs locally.
AMD MI50 32GB · llama.cpp · 120,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports a fork optimized for AMD gfx906 (MI50/MI60/Radeon VII) reaching 379.2 t/s prefill versus 332.3 t/s (+14.1%), 252.6 t/s versus 231.1 t/s on a 120k-context fill (+9.3%), and 13.6 t/s versus 13.5 t/s deep-context TG. DFlash acceptance is 0.691.
2× AMD MI50 32GB · llama.cpp
User benchmarks MTP KV cache quantization at Q8_0 against no quantization on Qwen3.6-27B-Q8_0 with llama.cpp. Setup uses tensor parallelism on 2xMI50 32GB. Wall time difference is negligible at roughly 0.14s with the quantized draft KV cache. Aggregate accept rate is 0.735-0.741.
Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.
User reports Kimi K2.6 at 264 t/s prompt processing and 9.7 t/s generation on 32x AMD MI50 32GB GPUs.