llamaperf

Muse 30B Glimmer

on NVIDIA RTX Pro 6000 Blackwell · vLLM · 131,072 ctx

NVIDIA hardware
Sep 7, 2026
Throughput
57.0 t/s gen
Quant
BF16

Summary

User reports Muse Glimmer 30B at ~57 t/s with speculative decoding using DFlash, against ~25 t/s without speculation, a 2.3x speedup. Setup is vLLM with six patches applied to the image. Mean acceptance length is ~2.5 tokens per verification step, with overall draft acceptance ~10%. Per-position acceptance is ~73% at position 0, ~40% at 1, ~15% at 2, and near zero past 5. The recipe's 3.1x figure was greedy decoding with K-quant on llama.cpp under different conditions. The predicted ceiling is ~65 t/s at 2.6 mean acceptance.