DeepSeek V4
M1 Ultra 128GB · LM Studio
- generation:
- 16.0 tokens/s
- quant:
- IQ3_XXS
User reports a patched engine improved speed from 5-6 to 15-16 tok/s. Setup uses the Unsloth UD-IQ3_XXS quant. The user notes a wired limit at 120GB.
Local model performance on your hardware
Find which open-weight LLMs fit in your GPU or Mac and compare the speeds people report on setups like yours.
Join the community
What runs on your hardware?
Pick your GPU or Mac, then read the speeds people reported on it, or estimate which models fit and how fast they run.
Free to use. No account needed. Memory estimates and community measurements are labelled separately.
Looking for a particular model?
Search for a model to see its reported speeds across GPUs and Macs.
Local model performance reports from the community
These are individual setups, not a controlled benchmark. Compare GPU count, quantization, context and offloading before comparing speeds. How to read a report →
Model: DeepSeek V4
Compare setup details
Exact recorded values. Context may be a configured limit; matching filters does not establish identical prompts, offloading or concurrency.
DeepSeek V4
3× NVIDIA RTX 3090 · llama.cpp · 1,000,000 ctx
- generation:
- 15.0 tokens/s
- prompt processing (prefill):
- 300.0 tokens/s
User reports 1M lossless context on 3x RTX 3090 with DDR5, at 15 t/s on cold start and 10 t/s steady. Setup is a llama.cpp fork with deepseek4 architecture support. A fourth RTX 3090 runs gemma 12b and flux2klein. Prefill starts at 300 t/s and degrades to 100 t/s after 64,000 tokens.
- prompt processing (prefill):
- 255.0 tokens/s
- quant:
- W8A8
User reports custom Turing CUDA kernels for W8A8 INT8 matmul, heterogeneous inference with 4x 11/22GB VRAM and 1TB system RAM, and computation-communication overlap for MoE routing. The work is open-sourced on GitHub.
DeepSeek V4 284B Flash
2× AMD Radeon AI PRO R9700 32GB · 16,384 ctx
- generation:
- 51.1 tokens/s
- quant:
- q2
User reports a Lucebox custom setup with an AMD Radeon AI PRO R9700 and Strix Halo 128GB running 3.63x faster than a single DGX Spark. Setup uses asymmetric parallelism: the R9700 handles the dense path, hot experts, cache and draft model, while the Strix Halo holds the other experts. The run is experimental, at q2 quant and 16k context. The user is working on KVFlash for 64k-128k.
Showing 1–4 of 4
Page 1 of 1
Community benchmarks snapshot
Records by GPU
Records by model
Qwen3.81215
Qwen3.6274
Gemma 4165
DeepSeek V4 Flash148
Qwen3.5116
Qwen371
other595
Records by engine
llama.cpp983
vLLM201
Strata132
MLX106
Ollama79
other407
Use cases
coding 685agentic 434long-context 323tool-use 189vision 126summarization 61math 51creative-writing 43multilingual 26rp 12text-generation 9reasoning 3
Median generation t/s by GPU
On Qwen3.8 27B at 4-bit, plain single-GPU runs. Full ranking