llamaperf

NVIDIA RTX 5090 Laptop 24GB

NVIDIA · 24GB · 11 reports

Is this your card?

On the NVIDIA RTX 5090 Laptop 24GB, Qwen3.8 27B at 4-bit, the model most run on it, generates a median of 33.5 tokens per second from 2 plain runs (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, the models most run on the NVIDIA RTX 5090 Laptop 24GB, each one size at one quant level, with the median generation speed of its plain runs (one device, one request, no speculative decoding):

Bars are medians on one scale, 0 to 107.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Engines people use on it: llama.cpp 4 (reports)

Run models on your NVIDIA RTX 5090 Laptop 24GB? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance on the NVIDIA RTX 5090 Laptop 24GB

These reports mix GPU counts and memory setups, so a model listed here may have needed extra cards or CPU offloading. The calculator estimates which models fit in 24 GB of VRAM. Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Tone: mixedNVIDIA hardware
generation:
300-500 tokens/s
quant:
NVFP4 (NVFP4)
tool-useagentic

User reports SenseNova-Loop-9B running at 300-500 tok/s on a laptop RTX 5090 in NVFP4. The model is a looped transformer with a small drafter, and the user says its architecture translates well to existing kernels. The user cannot yet confirm it holds up against Qwen3.8 27B NVFP4 and suspects their setup may be broken despite a calibration run; in their tool-eval-bench runs SNL-9B scored 92 vs 98 on the standard set and 73 vs 87 on hard mode, with 0 on tenant separation and policy application.

Nov 18, 2026
Tone: positiveNVIDIA hardware
generation:
107.0 tokens/s
prompt processing (prefill):
3706.5 tokens/s
quant:
Q4_K_M (GGUF)
kv:
Q8_0
codingagentic

User reports Qwen3-Coder-30B-A3B-Instruct at 107.0 tok/s generation and 3,706.5 tok/s prefill on an RTX 5090 Laptop 24GB in an MSI Raider A18. Setup is llama.cpp with CUDA, Q4_K_M GGUF (17.28 GiB), Q8_0 KV cache, FlashAttention on, 65,536-token context, all layers on the GPU, one slot. The figures come from a repository-analysis workload with an 8,638-token prompt and 371 generated tokens; the user notes the screenshot captures one strong interval, not an average across the audit.

Nov 17, 2026
Tone: positiveNVIDIA hardware
generation:
110.3 tokens/s
prompt processing (prefill):
3254.0 tokens/s
quant:
MLP-NVFP4 hybrid (GGUF)
agentic

User reports Muse Glimmer 30B at 110.3 tok/s greedy decode and 3,254 tok/s prefill on an RTX 5090 Laptop 24GB card. Setup is a patched llama.cpp with an MLP-NVFP4 hybrid quant, DFlash 2 speculative decoding at depth 15, and 128K context. A baseline llama.cpp build with unsloth UD-Q4_K_XL and Meta DFlash reached 80.6 tok/s decode and 1,430 tok/s prefill on the same 6,880-token prompt. A 100,554-token prompt prefilled at 2,367 tok/s, and an agent turn with a cached 46K prompt took 0.89 s. KLD vs Q8_0 rose to 0.053/0.030/0.034 (agent/English/Swedish) from 0.033/0.012/0.015.

Nov 16, 2026
NVIDIA hardware
generation:
30.0 tokens/s
quant:
NVFP4
codinglong-context

User reports about 30 t/s running Qwen3.8 Flash-Next in NVFP4 at 256k context on a laptop with a 16 GB Nvidia GPU and 96 GB DDR5 RAM. Setup details beyond the quant and context length are not stated. The figure is the user's current machine, cited while asking whether to buy a 24 GB VRAM / 128 GB RAM laptop or a 16 GB VRAM / 192 GB RAM laptop for local coding.

Nov 12, 2026
NVIDIA hardware
generation:
125.5 tokens/s
prompt processing (prefill):
2993.2 tokens/s
quant:
IQ2_XS (GGUF)

User reports Qwen3.8 Swift-1.5 Flash-Next at 125.5 tok/s generation and 2,993.2 tok/s prefill on an RTX 5090 Laptop 24 GB. Setup is PolyStrata with an IQ2_XS GGUF of about 63.5 GiB, configured for 131,072 tokens of context, with 128 GB of system RAM. The model is far larger than the card's 24 GB, so weights ran partly from system memory.

Oct 26, 2026
NVIDIA hardware
generation:
165.6 tokens/s
prompt processing (prefill):
9555.9 tokens/s
quant:
Q4_K_M (GGUF)

User reports 165.64 t/s generation and 9555.89 t/s prompt processing. Source evidence (verbatim): Source: https://blog.majid.info/llama-bench/golgotha.txt # NVIDIA GeForce RTX 5090 Laptop GPU 24GB: original Gemma4 E4B Q4_K_M llama-bench log First-hand author log linked from https://blog.majid.info/llama-bench/ (April 13, 2026 article). Measured hardware: NVIDIA GeForce RTX 5090 Laptop GPU 24GB. Gemma4 E4B Q4_K_M GGUF, llama.cpp e3f542d8 (4236), backend CUDA. pp512 9555.89 ±101.42 tokens/s; tg128 165.64 ±0.19 tokens/s. One device, ngl -1; 23983 MiB visible VRAM, not resident use. Laptop GPU, not desktop RTX5090. CPU and system RAM unknown. Command names --hf-repo unsloth/gemma-4-E4B-it-GGUF; exact tested quantization repository https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF ; tested revision unknown. Log reports 4.62 GiB model size and 7.52B counted parameters; size is not resident memory. Context depth, KV quantization, measured power, driver version and explicit speculation/concurrency declarations unreported. Raw source: https://blog.majid.info/llama-bench/golgotha.txt .

Oct 14, 2026
Tone: negativeNVIDIA hardware
generation:
37.0 tokens/s
quant:
Q4_K_XL (GGUF)
codingagentic

User reports Qwen3.8-27B with vision at about 37 tok/s on an RTX 5090 Laptop 24GB with 64GB RAM. Setup is Q4_K_XL at 100K context, tested with the opencode and pi harnesses. User also ran a two-laptop cluster of RTX 5080 Laptop 16GB machines: Qwen3.8-27B Q6_K fine-tuned for coding agents at 67 to 77 tok/s, Qwen3.8-27B Q5_K_XL at 53 tok/s, Qwen3-Coder-Next Q3_K_XL at 48 tok/s, and Qwen3.8-Flash-Next IQ4_XS at 19 tok/s. On the single 5090 laptop, Qwen3-Coder-Next Q3_K_XL reached 55 tok/s and Qwen3.8-Flash-Next with vision IQ4_XS at 262K context reached 20 tok/s. User says none of the local models can complete even minor tasks, filling context without output or producing nonsense, and that hosted GPT-6 and Opus 5.5 handle the same tasks without issue.

Oct 8, 2026

Qwen3.8 27B

NVIDIA RTX 5090 Laptop 24GB · llama.cpp · 65,536 ctx

Tone: mixedNVIDIA hardware
generation:
30.0 tokens/s
quant:
Q4_K_M (GGUF)
kv:
8bit
coding

User reports Qwen3.8 27B at a steady 30 t/s on an RTX 5090 Laptop 24GB. Setup is llama.cpp with Q4_K_M and 8-bit KV cache at 65k context, using 21 GB of VRAM. The user also tested Unsloth UD_Q5_K_XL at 27 t/s, and NInfer models reaching 82.54 t/s (8-bit, 132k context), 92.4 t/s (NVFP4, 32k context), and 86.63 t/s (NVFP4, 4-bit, 64k context). The user says the laptop is too slow for coding and recommends a DGX Spark instead.

Oct 7, 2026
Tone: mixedNVIDIA hardware

User reports a DIY Jev-like inference setup using unmodified open weight LLMs, evaluated on a 32,235-example benchmark. Qwen3.6 35B-A3B reached 75.5% accuracy at ~5.3 req/s on a laptop RTX 5090 24GB, with Qwen3 27B at 75.3% ~2.9 req/s and Qwen3-4B at 65.0% ~27 req/s. The approach uses boolean verification of candidate answers via true/false logits, batched through llama.cpp, with no NLI fine-tuning or classifier head. User notes the benchmark is not perfectly apples-to-apples and expresses skepticism about Jev hype.

Sep 20, 2026
Tone: mixedNVIDIA hardware
generation:
3.6 tokens/s
quant:
INT4_SYM (OpenVINO IR)

User reports Qwen3-14B Heretic at 3.56 t/s on an Intel AI Boost NPU37XX in an Acer Predator Helios 16 AI laptop with 32 GB RAM and an RTX 5090 Laptop 24GB. Setup is OpenVINO GenAI with the machine-made-Fibre INT4_SYM OpenVINO IR export, group_size=-1, all_layers=true, loaded directly via openvino_genai.LLMPipeline on NPU with no conversion. Load took 33.52 s and 96 output tokens took 26.98 s; peak RAM was about 15.6 GB with a 9.7 GB working set. User also benchmarked Qwen3-8B INT4 SYM on the same NPU at 6.72 t/s with 6.15 s load, and found Qwen3-30B-A3B MoE impractical due to host memory pressure during OpenVINO preparation.

Sep 18, 2026

Qwen3.6 27B

NVIDIA RTX 5090 Laptop 24GB · llama.cpp · 200,000 ctx

Tone: positiveNVIDIA hardware
quant:
IQ4_XS (GGUF)
kv:
Q8
rating:
5/5
codingtool-use

User reports Qwen 3.6 27B is excellent for pyspark/python and data transformation debugging, running on an ASUS ROG Strix SCAR 18 with an RTX 5090 laptop (24 GB VRAM) and 64 GB DDR5 RAM. Setup is llama.cpp with the IQ4_XS quant at 200k context and a Q8_0 KV cache. The user initially tried q4_k_m at q4_0. No tokens/sec is reported. The user is cancelling cloud subscriptions due to local performance.

Apr 28, 2026

Get a weekly email of new NVIDIA RTX 5090 Laptop 24GB reports and newly released models that fit it.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3-Coder 30B (3B active)
NVIDIA RTX 5090 Laptop 24GB
Q4_K_M
llama.cpp
65,536107.0 tokens/s
Muse 30B Glimmer
NVIDIA RTX 5090 Laptop 24GB
MLP-NVFP4 hybrid
llama.cpp
128,000110.3 tokens/s
Qwen3.8 125B (6B active) Flash-Next
NVIDIA RTX 5090 Laptop 24GB
NVFP4
Engine not reported
262,14430.0 tokens/s
Qwen3.8 125B (6B active) Swift-1.5 Flash-Next
NVIDIA RTX 5090 Laptop 24GB
IQ2_XS
PolyStrata
131,072125.5 tokens/s
Gemma 4 8B E4B
NVIDIA RTX 5090 Laptop 24GB
Q4_K_M
llama.cpp
Not reported165.6 tokens/s
Qwen3.8 27B
NVIDIA RTX 5090 Laptop 24GB
Q4_K_XL
Engine not reported
100,00037.0 tokens/s

Frequently asked

How fast is the NVIDIA RTX 5090 Laptop 24GB for local LLMs?

On the NVIDIA RTX 5090 Laptop 24GB, Qwen3.8 27B at 4-bit, the model most run on it, generates a median of 33.5 tokens per second from 2 plain runs (community reports, as of 22 Nov 2026). Also reported: Qwen3 14B at 4-bit (3.6 t/s, one plain run); Qwen3-Coder 30B · 3B active at 4-bit (107.0 t/s, one plain run).

What is the biggest model reported running on the NVIDIA RTX 5090 Laptop 24GB?

The largest model with a plain generation run on the NVIDIA RTX 5090 Laptop 24GB is Qwen3-Coder 30B · 3B active at 4-bit (107.0 t/s, one plain run), as of 22 Nov 2026. Plain means one device, one request, no speculative decoding, so a run split across several cards does not count.