llamaperf

Muse 27.9B Glimmer 30B

on 2× NVIDIA RTX PRO 5000 Blackwell (one 72GB card and one 48GB card) · llama.cpp

Oct 13, 2026
Throughput
63.0 t/s gen · 3241.8 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q8_0 (GGUF)
KV cache
F16
Flash Attention
on
System RAM
96 GB

Summary

User reports Muse Glimmer 30B at 63 t/s generation and 3241.8 t/s prompt processing on two RTX PRO 5000 Blackwell cards, 72 GB and 48 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K/V cache, flash attention on, batch 2048 and microbatch 512, tensor split 1/1 across the two GPUs. The 63 t/s is TG128 and the 3241.8 t/s is PP512 from separate synthetic tests, not a combined request; the author reports TG reproducibility around 1% and PP drift up to about 10% between sessions.