llamaperf

NVIDIA RTX 4060 Laptop 8GB reports

6 reports. These are individual setups, not a controlled benchmark. NVIDIA RTX 4060 Laptop 8GB speeds and models →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

NVIDIA hardware
generation:
28.0 tokens/s
prompt processing (prefill):
31.2 tokens/s
quant:
PTQ1_0 (GGUF)

User reports Bonsai 2 27B at 28.0 tok/s on an RTX 4060 Laptop 8GB, with prompt reading at 31.2 tokens/s. Setup is the Kindi 0.3.0 image with a PTQ1_0 GGUF at 24,576 context, leaving 1,367 MiB free. Decode falls from 28.0 tok/s at position 100 to 25.2 tok/s at 24,000; a verify run passed token-exact against llama.cpp at 29.3 tokens/s.

Oct 31, 2026
Tone: mixedNVIDIA hardware
generation:
1.0 tokens/s
kv:
q8_0
agentic

User asks how to run LFM3 1.7B at high concurrency locally, using llama.cpp with q8_0 KV cache and --parallel 16 --kv-unified on an RTX 4060 Laptop with 8 GB VRAM and 16 GB system RAM. User reports that beyond about 6 concurrent sequences throughput stops scaling, and that starting a new request drops every other instance to about 1 tok/s until that request finishes prefill. No single-stream throughput figure is given; the 1 tok/s is a per-stream rate under concurrent load.

Oct 28, 2026
Tone: positiveNVIDIA hardware
generation:
60.0 tokens/s

User reports Gemma 4 26B MoE at 60+ tok/s on an RTX 4060 Laptop, up from 40 tok/s with standard llama.cpp. Setup is llama.cpp with MTP speculative decoding plus a custom DeepSeek-inspired hot-expert caching scheme that keeps frequently used experts on the GPU. The user asks for further performance ideas without changing the model.

Oct 14, 2026
NVIDIA hardware
generation:
37.7 tokens/s
prompt processing (prefill):
1828.8 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3.5 9B at 37.7 t/s generation and 1828.8 t/s prompt processing on an NVIDIA RTX 4060 Laptop 8GB. Setup is llama.cpp with Q4_K_M and full GPU offload via -ngl 99, run through native llama-bench CUDA on a Lenovo LOQ 15IRX9 with 24GB system RAM. The figures come from separate synthetic tests, pp512 and tg128, not a single 512-token request followed by a 128-token completion; context allocation and KV precision are not given.

Oct 12, 2026

Cyber-Tiel 35B (3B active) Coder

NVIDIA RTX 4060 Laptop 8GB · llama.cpp · 131,072 ctx

NVIDIA hardware
generation:
23.0 tokens/s
prompt processing (prefill):
35.0 tokens/s
quant:
UD-IQ3_XXS (GGUF)
kv:
q4_0
coding

User reports Cyber-Tiel Coder 35B-A3B at 23 t/s decode and 35 t/s prefill on an RTX 4060 Laptop 8GB with 16 GB system RAM, at 131,072 context. Setup is llama.cpp with the UD-IQ3_XXS GGUF, q4_0 KV cache, flash attention, 30 MoE layers offloaded to CPU, and MTP speculative decoding with 2 draft tokens. Figures are at 4K context and depend on free RAM for mmap caching; a 13-prompt benchmark table is promised later.

Sep 23, 2026
Tone: positiveNVIDIA hardware
generation:
39.0 tokens/s
agentic

User reports Qwen3.6 35B at 39 t/s on an 8 GB RTX 4060 laptop using the FreeToken engine. User also reports DeepSeek-V4-Flash 284B at 22-25 t/s on an RTX 5090 and GLM-5.2 753B at 15 t/s on an RTX PRO 6000.

Sep 7, 2026