llamaperf

NVIDIA RTX 5060 Laptop 8GB reports

5 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX 5060 Laptop 8GB speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: positiveNVIDIA hardware
generation:
30-40 tokens/s
prompt processing (prefill):
800-970 tokens/s
quant:
TQ1_0 (GGUF)
codingagenticvision

User reports running Qwen3.8 Flash-Next (125B-A6B) with the Gyro S TQ1_0 GGUF on an RTX 5060 Laptop with 8 GB VRAM and 32 GB RAM, getting PP 800-970 t/s and TG 30-40 t/s at 262K context with the Chat configuration. Setup is Strata with experts mmapped to SSD, KV cache split between VRAM and RAM, and Vision run on the CPU; MTP is disabled. At 131K context the Chat configuration gives PP 930-1,040 t/s and TG 30-45 t/s, while the Agentic configuration gives PP 520-740 t/s and TG 30-40 t/s at 262K and PP 670-890 t/s and TG 30-45 t/s at 131K. The user compares this with Qwen 3.6 35B A3B in llama.cpp at PP 630-680 t/s and TG 27-35 t/s, and says the model replaced all their coding agents.

Oct 26, 2026
Tone: positiveNVIDIA hardware
generation:
43.0 tokens/s

User reports Qwen3.6-35B-A3B at a little over 43 tokens/s on an RTX 5060 Laptop 8GB in a Lenovo Legion 5 with a Ryzen AI 7 350 and 32GB DDR5-5600. Setup is llama.cpp with the model offloaded across the 8GB VRAM and system RAM; around 30K context looks promising. User is also building PolyStrata to try for more performance, with no numbers yet.

Oct 14, 2026
NVIDIA hardware
generation:
29.3 tokens/s
quant:
PTQ1_0 (GGUF)
math

User reports Ternary Bonsai 2 27B at 29.27 t/s on an RTX 5060 Laptop 8 GB under Windows, using a Prism llama.cpp build. Setup is llama.cpp with PTQ1_0 (5.53 GiB), temp 0, seed 42, a 2048 reasoning budget and a 3072 token cap on 50 MATH-500 questions. A −2 logit bias on "wait", "maybe" and "perhaps" (nine token ids) dropped accuracy from 44/50 to 43/50 and raised average tokens from 845.2 to 872.3; the biased run measured 29.28 t/s. The user calls it one deterministic data point, not proof.

Oct 3, 2026
Tone: positiveNVIDIA hardware
prompt processing (prefill):
300-400 tokens/s
codingagentic

User reports Ornith 1.5 35B-A3B at 30-40 t/s generation and 300-400 t/s prompt processing on an RTX 5060 Laptop 8GB with 32GB DDR5 RAM. Setup uses llama.cpp via OpenCode, with generation at 30 t/s for 100k-140k context and up to 40 t/s below 100k context. User also ran Qwen 3.8 27B on the same hardware and compares the two models for coding tasks, noting Ornith handles contained tasks well but struggles with complex multi-file changes.

Sep 26, 2026

Gemma 4 12B

NVIDIA RTX 5060 Laptop 8GB · llama.cpp · 4,096 ctx

NVIDIA hardware
generation:
40.0 tokens/s
prompt processing (prefill):
700-900 tokens/s
quant:
Q6_K (GGUF)
kv:
F16

User reports Gemma 4 12B at 40.0 t/s on an RTX 5060 Laptop GPU with 8 GB VRAM, using MTP speculative decoding with a Q6_K draft head and F16 KV cache at 4096 context. Setup is llama.cpp with mainline draft-mtp support (commit b9193 or later), an imatrix-guided per-tensor fit-to-VRAM quant at 6.18 GB, 48 layers, parallel 1. Prefill is 700 to 900 t/s. Without MTP the same setup reaches 27.4 t/s. With a Q8 KV cache it reaches 40.4 t/s. In real chat use the user sees 42 or more t/s on fresh context, dropping to 33 to 37 t/s with 16k of context filled.

Sep 24, 2026