Choosing a model · · 7 min read
Small local LLM: how to pick one for modest hardware
How to choose a small local LLM for a laptop or an 8 GB graphics card: read the model card, check the licence, then test it on your own tasks.
Practical ways to choose a local LLM, measure its speed and improve your setup.
Choosing a model · · 7 min read
How to choose a small local LLM for a laptop or an 8 GB graphics card: read the model card, check the licence, then test it on your own tasks.
Measurement · · 6 min read
We compared the calculator's tokens-per-second estimates with what people measured on the same card, model and quant. Where it lands, and where it misses.
Measurement · · 5 min read
A small test log that shows whether an update or a settings change made your local LLM setup better, and lets you roll back when it didn't.
Hardware planning · · 5 min read
Before you buy a second graphics card, work out whether you need more memory or faster answers, and check that your runtime can use it.
Coding workflows · · 5 min read
Try a local coding model on small changes from your own repo, and judge it by how long it takes to reach a patch you would merge.
Quantization · · 5 min read
A plain A/B test of two quantizations of one model, run on your own prompts and pass checks, to see whether the smaller file keeps the quality you need.
Mac setups · · 6 min read
How much of a Mac's unified memory a local model can use, and how to test a setup with your everyday apps open before you pay for more memory.
Troubleshooting · · 6 min read
A long prompt that feels slow is either slow to read or slow to answer. How to tell which with a small timing test before you change a single setting.
Hardware planning · · 6 min read
Picking a local LLM for a 12 GB card: what fits at each quant, how much room context takes, and how to tell when part of the model spilled to the CPU.
Reading benchmarks · · 10 min read
How Strata runs Qwen3.8-Flash-Next, a 125B model, on a 12 or 16 GB card, what owners measure with it, and how it compares with llama.cpp on the same PC.
Hardware planning · · 5 min read
AMD's Ryzen AI Max 400 (Gorgon Halo) lifts Strix Halo to 192 GB with barely more bandwidth, so it loads bigger local models without running them faster.
Mac setups · · 7 min read
What owners measured on the new Mac Studio with M5 Max and M5 Ultra, how much faster the Ultra really is, and which memory size to buy for local models.
Hardware planning · · 7 min read
NVIDIA's RTX Spark brings 128 GB of unified memory to laptops. DGX Spark's measured speeds show what it will run and what NVIDIA hasn't said yet.
Measurement · · 5 min read
Get a clean output speed from Ollama's own response fields, and keep model loading and prompt reading out of the number you publish.
Reading benchmarks · · 6 min read
Two people with the same GPU can report wildly different local LLM speeds. Line these settings up before you believe either tokens per second figure.
Choosing a model · · 6 min read
Pick a local model by whether it fits your memory and gets your real tasks right, then check its speed against reports from machines like yours.
Choosing a model · · 5 min read
Which engine should run your local model? Pick by who sends it requests and which model files it reads on your hardware, then test two on one prompt.
Measurement · · 5 min read
Log your GPU's power draw with nvidia-smi, line it up with a timed run and work out the energy each answer costs, while avoiding the usual measuring traps.
Hardware planning · · 5 min read
How Ollama and llama.cpp's server answer several people at once with parallel slots, what each costs in memory, and how per-user speed falls as output rises.
Reading benchmarks · · 5 min read
A draft model or built-in MTP can make local answers arrive faster. When speculative decoding helps, how to turn it on, and why your speed report should say so.
Hardware planning · · 5 min read
No graphics card? You can still run a local model on CPU only. Memory bandwidth sets your speed, and a quick test shows whether you need a GPU at all.
Measurement · · 6 min read
Time to first token is the wait before any text appears. Measure it from a streaming request and keep model loading and prompt caching out of it.
Choosing a model · · 6 min read
A mixture-of-experts model uses only a few experts per token, yet all of them have to sit in memory. What that means for MoE memory and speed on your hardware.
Troubleshooting · · 5 min read
When a model is too big for VRAM, llama.cpp and Ollama split its layers between GPU and CPU. How to read that split, set -ngl and decide if it pays.
Hardware planning · · 6 min read
Every token in your context keeps its keys and values in memory. How the KV cache grows with context length, and how to size and shrink it in Ollama.
Quantization · · 6 min read
Q4_K_M, Q6_K, Q8_0, IQ4_XS: what each part of a GGUF file name tells you about bits per weight, file size and the memory you'll need.