llamaperf

NVIDIA RTX 4060 Laptop 8GB

NVIDIA · 8GB · 6 reports

Is this your card?

On the NVIDIA RTX 4060 Laptop 8GB, Qwen3.5 9B at 4-bit, the model most run on it, generates a median of 37.7 tokens per second from one plain run (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the NVIDIA RTX 4060 Laptop 8GB, each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

  • Qwen3.5 9B4-bit · PP 1828.8 t/s, one run
    37.7 t/s · one run

Bars are medians on one scale, 0 to 37.7 tokens/s. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: llama.cpp 4 (reports) · FreeToken 1 (reports)

Run models on your NVIDIA RTX 4060 Laptop 8GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA RTX 4060 Laptop 8GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 8 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

NVIDIA hardware
generation:
28.0 tokens/s
prompt processing (prefill):
31.2 tokens/s
quant:
PTQ1_0 (GGUF)

User reports Bonsai 2 27B at 28.0 tok/s on an RTX 4060 Laptop 8GB, with prompt reading at 31.2 tokens/s. Setup is the Kindi 0.3.0 image with a PTQ1_0 GGUF at 24,576 context, leaving 1,367 MiB free. Decode falls from 28.0 tok/s at position 100 to 25.2 tok/s at 24,000; a verify run passed token-exact against llama.cpp at 29.3 tokens/s.

Oct 31, 2026
Tone: mixedNVIDIA hardware
generation:
1.0 tokens/s
kv:
q8_0
agentic

User asks how to run LFM3 1.7B at high concurrency locally, using llama.cpp with q8_0 KV cache and --parallel 16 --kv-unified on an RTX 4060 Laptop with 8 GB VRAM and 16 GB system RAM. User reports that beyond about 6 concurrent sequences throughput stops scaling, and that starting a new request drops every other instance to about 1 tok/s until that request finishes prefill. No single-stream throughput figure is given; the 1 tok/s is a per-stream rate under concurrent load.

Oct 28, 2026
Tone: positiveNVIDIA hardware
generation:
60.0 tokens/s

User reports Gemma 4 26B MoE at 60+ tok/s on an RTX 4060 Laptop, up from 40 tok/s with standard llama.cpp. Setup is llama.cpp with MTP speculative decoding plus a custom DeepSeek-inspired hot-expert caching scheme that keeps frequently used experts on the GPU. The user asks for further performance ideas without changing the model.

Oct 14, 2026
NVIDIA hardware
generation:
37.7 tokens/s
prompt processing (prefill):
1828.8 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3.5 9B at 37.7 t/s generation and 1828.8 t/s prompt processing on an NVIDIA RTX 4060 Laptop 8GB. Setup is llama.cpp with Q4_K_M and full GPU offload via -ngl 99, run through native llama-bench CUDA on a Lenovo LOQ 15IRX9 with 24GB system RAM. The figures come from separate synthetic tests, pp512 and tg128, not a single 512-token request followed by a 128-token completion; context allocation and KV precision are not given.

Oct 12, 2026

Cyber-Tiel 35B (3B active) Coder

NVIDIA RTX 4060 Laptop 8GB · llama.cpp · 131,072 ctx

NVIDIA hardware
generation:
23.0 tokens/s
prompt processing (prefill):
35.0 tokens/s
quant:
UD-IQ3_XXS (GGUF)
kv:
q4_0
coding

User reports Cyber-Tiel Coder 35B-A3B at 23 t/s decode and 35 t/s prefill on an RTX 4060 Laptop 8GB with 16 GB system RAM, at 131,072 context. Setup is llama.cpp with the UD-IQ3_XXS GGUF, q4_0 KV cache, flash attention, 30 MoE layers offloaded to CPU, and MTP speculative decoding with 2 draft tokens. Figures are at 4K context and depend on free RAM for mmap caching; a 13-prompt benchmark table is promised later.

Sep 23, 2026
Tone: positiveNVIDIA hardware
generation:
39.0 tokens/s
agentic

User reports Qwen3.6 35B at 39 t/s on an 8 GB RTX 4060 laptop using the FreeToken engine. User also reports DeepSeek-V4-Flash 284B at 22-25 t/s on an RTX 5090 and GLM-5.2 753B at 15 t/s on an RTX PRO 6000.

Sep 7, 2026

Get a weekly email of new NVIDIA RTX 4060 Laptop 8GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Bonsai 2 27B
NVIDIA RTX 4060 Laptop 8GB
PTQ1_0
Kindi
24,57628.0 tokens/s
LFM3 1.7B
NVIDIA RTX 4060 Laptop 8GB
Not reported
llama.cpp
Not reported1.0 tokens/s
Gemma 4 26B
NVIDIA RTX 4060 Laptop 8GB
Not reported
llama.cpp
Not reported60.0 tokens/s
Qwen3.5 9B
NVIDIA RTX 4060 Laptop 8GB
Q4_K_M
llama.cpp
Not reported37.7 tokens/s
Cyber-Tiel 35B (3B active) Coder
NVIDIA RTX 4060 Laptop 8GB
UD-IQ3_XXS
llama.cpp
131,07223.0 tokens/s
Qwen3.6 35B (3B active)
NVIDIA RTX 4060 Laptop 8GB
Not reported
FreeToken
Not reported39.0 tokens/s

Frequently asked

How fast is the NVIDIA RTX 4060 Laptop 8GB for local LLMs?

On the NVIDIA RTX 4060 Laptop 8GB, Qwen3.5 9B at 4-bit, the model most run on it, generates a median of 37.7 tokens per second from one plain run (community reports, as of 22 Nov 2026).