Muse

reported speed:

35.4 tokens/s generation · 355.4 tokens/s prompt processing

quant:

Q6_K_XL (GGUF)

kv:

f16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmark results for Muse Glimmer 30B on AMD v620 GPUs. Q6 1 GPU: PP 355.38 t/s, gen 35.38 t/s. Q6 tensor split 2 GPUs: PP 472.27 t/s, gen 36.32 t/s. Q8 tensor split 2 GPUs: PP 550.04 t/s, gen 26.55 t/s. Uses speculative decoding with dflash draft model.

reported speed:

173.0 tokens/s generation · 2654.0 tokens/s prompt processing

quant:

UD-Q6_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Speculative decoding with DFlash. Peak generation 287 t/s, average 173 t/s. Prompt processing short 1000-2100 t/s, long 2654 t/s. Draft acceptance 63.3-97.5%, mean accepted draft length up to 14 tokens. KV cache reuse 99.9%. Compared to Qwen 3.6 27B: no Chinese language-mixing bugs, no overthinking loops, concise responses.

reported speed:

20.0 tokens/s generation · 308.0 tokens/s prompt processing

quant:

UD-Q2-K-XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Ran Muse Glimmer 30B with UD-Q2-K-XL quant and DFlash speculative decoding on RX 7600 XT 16GB via llama.cpp with ROCm. Achieved ~308 t/s prompt processing and ~20 t/s generation. Successfully completed a coding task.

reported speed:

84.6 tokens/s generation

quant:

Q5_K_M

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Speculative decoding with DFlash. Best tuned 256K run: 84.64 tok/s, 38.67% acceptance. Regular decoding: 17.98 tok/s. NVFP4 hybrid reached 94.36 tok/s but worse perplexity. Q5_K_M chosen as best. Context fill of 262,116 tokens completed without OOM, using 22,920/24,467 MiB VRAM, decode at 21.56 tok/s.

reported speed:

30.0 tokens/s generation

quant:

EXL3-SC 3.00bpw H4 (exl2)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Also tried Qwen 3.8 27B at SC2.20bpw H3, but prefers Unsloth UD_Q4_K_XL for coding.

reported speed:

94.0 tokens/s generation · 1400.0 tokens/s prompt processing

quant:

Q4_K_XL (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Muse Glimmer fits on a single RTX 3090 with full context + DFlash + mmproj at Q4_K_XL, unlike Qwen3.6-27B and Gemma-4-31B. Fits in about 22-23GB VRAM. Generation speed ranges from 64 to 124 tok/s depending on prose vs code; average reported as 94 tok/s. Prompt processing about 1400 tok/s. Compared to Qwen3.6-27B and Gemma-4-31B which are borderline unusable on the same GPU.

reported speed:

18.0 tokens/s generation

quant:

Q4_K_XL (GGUF)

kv:

q4_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Model fits on single RTX 5060 Ti 16GB with 131k context using Q4 KV cache. Q8 KV cache allows ~90k context. Only GGUF weights loaded, no dflash or mmproj.

reported speed:

233.4 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Mentions 256k context reachable on 24GB, but the benchmark was on 5090. The model name 'Glimmer' is not in canonical list but extracted as raw.

reported speed:

253.0 tokens/s generation

quant:

UD-Q5_K_M

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmarked Muse Glimmer 30B on RTX 5090 with 262k context. Achieved 253 t/s on code patch workload using PR #26842 + DFlash. Mentions Meta's published 233 t/s. Uses llama-server with dflash draft model.

reported speed:

125.0 tokens/s generation

quant:

Q5_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User tested Muse-Glimmer with dflash on RTX 5090, observed unusual reasoning traces (disorganized, repetitive, self-referential 'we', policy mentions). Generation speed ~90-160 tok/s depending on task. Compared reasoning to Gemini models, contrasting with Qwen 3.6. Questioned whether reasoning helps or hinders. Also tested with a coding prompt (Flappy Bird HTML game).

reported speed:

57.0 tokens/s generation

quant:

BF16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Speculative decoding with DFlash on Muse Glimmer 30B. Six patches needed to vLLM image. Without speculation ~25 tok/s, with DFlash ~57 tok/s (2.3x). Mean acceptance length ~2.5 tokens per verification step. Overall draft acceptance ~10%. Per-position acceptance: ~73% at pos 0, ~40% at 1, ~15% at 2, near zero past 5. Recipe's 3.1x was greedy decoding with K-quant on llama.cpp, different conditions. Predicted ceiling ~65 tok/s at 2.6 mean acceptance.

reported speed:

25.0 tokens/s generation

quant:

UD-Q2_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Running in-browser with custom WebGPU kernels, speed comparable to llama.cpp.

reported speed:

18.0 tokens/s generation

quant:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Speculative decoding on Apple Silicon. 8-bit model: 8.2 tok/s baseline to 18-26 tok/s with speculative decoding. Math 3.27x, code 2.5x, chat 2.22x. Output byte-identical. 4-bit ~1.7x at ~25 tok/s, needs ~18GB. 8-bit peaks ~40GB, needs 48GB Mac. Meta's DFlash numbers on Mac: 1.5x (M4 Max) / 1.8x (M5 Max) on 4-bit.

reported speed:

8.2 tokens/s generation

quant:

8-bit

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Speculative decoding project mlx-dspark. 8-bit model: 8.2 tok/s baseline, 18-26 tok/s with speculative decoding. Speedups: math 3.27x, code 2.5x, chat 2.22x. 4-bit: ~1.7x at ~25 tok/s, needs ~18GB. 8-bit peaks ~40GB, requires 48GB Mac. Output byte-identical to normal decoding.

reported speed:

35.0 tokens/s generation

quant:

Q8_0 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmark comparing TensorSharp vs llama.cpp on RTX PRO 6000 Blackwell. Plain text generation decode ~35 tok/s at 60 prompt tokens. Also tested DFlash speculative decoding and 2x RTX PRO 4000 Blackwell 24GB with tensor parallelism.

Profile v2.2 optimizer for vLLM. Achieved 421 tok/s at 25k ctx on RTX 5090 with muse-glimmer 30B, no DFlash spec decoding. TTFT 224ms (p95 500ms), TPOT 23ms. Cost reduced from $3.41 to $0.65 per 1M output tokens. Energy 4.72 to 1.08 J/tok. Mentions SWE-Bench agents.

reported speed:

37.0 tokens/s generation · 390.0 tokens/s prompt processing

quant:

K-Quant-Dynamic (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Tested retrieval at 832K tokens with 3/3 success. DFlash spec decode ~3x speedup. RPC split slower than single-node.