llamaperf

Muse 30B Glimmer

on NVIDIA RTX 5090 Laptop 24GB · llama.cpp · 128,000 ctx

Tone: positiveNVIDIA hardware
Nov 16, 2026
Throughput
110.3 t/s gen · 3254.0 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
MLP-NVFP4 hybrid (GGUF)
VRAM reported
24 GB

Use cases

agentic

Summary

User reports Muse Glimmer 30B at 110.3 tok/s greedy decode and 3,254 tok/s prefill on an RTX 5090 Laptop 24GB card. Setup is a patched llama.cpp with an MLP-NVFP4 hybrid quant, DFlash 2 speculative decoding at depth 15, and 128K context. A baseline llama.cpp build with unsloth UD-Q4_K_XL and Meta DFlash reached 80.6 tok/s decode and 1,430 tok/s prefill on the same 6,880-token prompt. A 100,554-token prompt prefilled at 2,367 tok/s, and an agent turn with a cached 46K prompt took 0.89 s. KLD vs Q8_0 rose to 0.053/0.030/0.034 (agent/English/Swedish) from 0.033/0.012/0.015.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Muse 30B at 4-bit by hardware

Bars are medians on one scale, 0 to 30.0 tokens/s. Press ▶ to watch text arrive at a row's speed.

Models most run on the NVIDIA RTX 5090 Laptop 24GB

Bars are medians on one scale, 0 to 107.0 tokens/s. Each dot is one run and opens its report. Press ▶ to watch text arrive at a row's speed.

Everything reported on the NVIDIA RTX 5090 Laptop 24GB →Every Muse report →Estimate Muse on this card →