llamaperf

Qwen3.8 27B

on 3× AMD MI50 16GB · llama.cpp · 262,144 ctx

Tone: positiveAMD hardware
Sep 26, 2026
Throughput
30.1 t/s gen
Quant
Q8_0 (GGUF)
KV cache
F16
Flash Attention
on
VRAM reported
49 GB

Use cases

agenticlong-context

Summary

User reports Qwen3.8-27B Q8_0 at about 30 tok/s decode on 3x AMD Instinct MI50 16GB (gfx906) with 262,144 configured context. Setup is llama.cpp with HIP/ROCm 6.2.4, RCCL, tensor split mode, F16 KV cache, flash attention on, and about 43.7 GiB aggregate VRAM use. At 216k occupied context decode was 18.24 tok/s with 187.75 tok/s prompt throughput. With HIP graphs enabled, a tiny request after a ~236k stress run stayed at 8.74 tok/s until the server was restarted; disabling graphs (GGML_CUDA_DISABLE_GRAPHS=1) cost about 3% at short context but restored ~30 tok/s immediately after the long-context run.