llamaperf

Unknown family 26B (4B active)

on AMD Radeon 780M · llama.cpp · 16,384 ctx

Tone: mixedAMD hardware
Nov 16, 2026
Throughput
25.0 t/s gen
Quant
Q4 QAT (GGUF)
Flash Attention
on
System RAM
64 GB

Use cases

agenticlong-context

Summary

User reports a fully local Italian voice assistant on a mini PC with a Radeon 780M iGPU and 64GB DDR5 shared memory, reaching a median ~2.2s from end of speech to first audio. The LLM is a 26B-A4B MoE at Q4 QAT running in llama.cpp on Vulkan at ~25 tok/s, with 16k context, one slot, flash attention and cache reuse. Prefill of the turn costs ~1.0s (~0.35s fixed per request plus ~3ms/token), ASR ~0.75s and first sentence plus TTS first chunk ~0.55s. The user puts the calculated floor on this box at ~1.2-1.5s. Smaller dense models were benchmarked and rejected for factual fidelity, and KV-cache warming during speech was net neutral-to-negative on one slot. The user asks what else could be done before buying a 24GB dGPU.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Models most run on the AMD Radeon 780M

Bars are medians on one scale, 0 to 25.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the AMD Radeon 780M →