llamaperf

The llamaperf blog

Practical ways to choose a local LLM, measure its speed and improve your setup.

Measurement · · 6 min read

How accurate is our local LLM speed calculator?

We compared the calculator's tokens-per-second estimates with what people measured on the same card, model and quant. Where it lands, and where it misses.

Troubleshooting · · 6 min read

Why long prompts can make a local LLM feel slow

A long prompt that feels slow is either slow to read or slow to answer. How to tell which with a small timing test before you change a single setting.

Hardware planning · · 6 min read

Choosing a local LLM with 12 GB of VRAM

Picking a local LLM for a 12 GB card: what fits at each quant, how much room context takes, and how to tell when part of the model spilled to the CPU.

Reading benchmarks · · 10 min read

Strata and Qwen3.8-Flash-Next on one gaming GPU

How Strata runs Qwen3.8-Flash-Next, a 125B model, on a 12 or 16 GB card, what owners measure with it, and how it compares with llama.cpp on the same PC.

Measurement · · 5 min read

How to measure tokens per watt on your local LLM

Log your GPU's power draw with nvidia-smi, line it up with a timed run and work out the energy each answer costs, while avoiding the usual measuring traps.

Hardware planning · · 5 min read

Running a local LLM on CPU only: what to expect

No graphics card? You can still run a local model on CPU only. Memory bandwidth sets your speed, and a quick test shows whether you need a GPU at all.

Choosing a model · · 6 min read

MoE models: total vs active parameters

A mixture-of-experts model uses only a few experts per token, yet all of them have to sit in memory. What that means for MoE memory and speed on your hardware.

Hardware planning · · 6 min read

Why a longer context needs more memory

Every token in your context keeps its keys and values in memory. How the KV cache grows with context length, and how to size and shrink it in Ollama.