Muse

generation:

7.3 tokens/s

prompt processing (prefill):

234.1 tokens/s

quant:

UD-Q8_K_XL (GGUF)

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse-Glimmer-30B at 7.3 t/s generation and 234.1 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory. Setup is llama.cpp with the Unsloth UD-Q8_K_XL quant, full offload, batch 2048 and flash attention, run as separate llama-bench pp512 and tg128 microbenchmarks rather than one end-to-end request. The main table figure is rounded; the source narrative gives approximately PP234 and TG7.25. A separate DFlash server test reports 13.53 t/s, which is not tg128 and is not part of this row.

generation:

28.5 tokens/s

prompt processing (prefill):

253.9 tokens/s

quant:

UD-Q4_K_XL (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse-Glimmer-30B at 28.5 t/s generation and 253.9 t/s prompt processing on a Mac Studio M4 Max with 128 GB unified memory. Setup is llama.cpp b10794 with UD-Q4_K_XL and F16 K/V cache, flash attention, 99 GPU layers requested, single sequence. Figures are separate 512-token prefill and 128-token generation microbenchmarks at prefix depth 0; at depth 4096 generation is 27.82 t/s and at 16384 it is 27.07 t/s.

generation:

13.2 tokens/s

prompt processing (prefill):

370.9 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

flash attention:

on

mtp (multi-token prediction):

off

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at 13.2 t/s generation and 370.9 t/s prompt processing on an AMD Strix Halo 128GB machine. Setup is llama.cpp build 11146 with Q4_K_M, F16 K/V cache, ngl 999, FA1, batch 2048 and ubatch 512, one Vulkan device. The headline pair is pass 1 pp512/tg128 at prepared depth 0, r10; tg128 at prepared depth 8192 measured 13.0 t/s.

generation:

8.6 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Muse at 8.6 t/s on an Apple Mac mini M6 with 32 GB unified memory. Setup is LM Studio with an unknown quantization and a loaded context of 32,768 tokens on one machine. The 8.6 t/s is the median of three English-prose runs, client-observed streaming decode throughput rather than a pooled or peak rate; the JSON task passed 0 of 3 automatic checks.

generation:

63.0 tokens/s

prompt processing (prefill):

3241.8 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at 63 t/s generation and 3241.8 t/s prompt processing on two RTX PRO 5000 Blackwell cards, 72 GB and 48 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K/V cache, flash attention on, batch 2048 and microbatch 512, tensor split 1/1 across the two GPUs. The 63 t/s is TG128 and the 3241.8 t/s is PP512 from separate synthetic tests, not a combined request; the author reports TG reproducibility around 1% and PP drift up to about 10% between sessions.

generation:

38.5 tokens/s

prompt processing (prefill):

2677.5 tokens/s

quant:

Q8_0 (GGUF)

kv:

F16

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at 38.5 t/s generation and 2677.5 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72 GB. Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048 and microbatch 512, whole model on GPU0. The 38.5 t/s is TG128 and the 2677.5 t/s is PP512, each the mean of five repetitions; the author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.

generation:

26.7 tokens/s

prompt processing (prefill):

269.0 tokens/s

quant:

int4hu block32 symmetric

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse at 26.7 t/s generation and 269 t/s prompt processing on an M4 Max 128GB. Setup is Apple's coreai-pipelined GPU engine with int4hu block32 symmetric quantization, text decoder only, 16.35 GB bundle size. The source table pairs 269.0 tok/s prompt with 26.69 tok/s decode under llm-benchmark at 512 prompt / 1024 generation, 3 trials; aggregation, cache state and batch configuration for that row are not stated.

generation:

42.0 tokens/s

prompt processing (prefill):

684.9 tokens/s

quant:

IQ4_XS (GGUF)

kv:

Q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Glimmer 30B at 41.97 t/s generation and 684.86 t/s prompt processing on an RTX 3060 12GB plus RTX 3070 8GB. Setup is llama.cpp with IQ4_XS weights, Q8_0 KV cache, 65,536-token context, Flash Attention on, and a DFlash draft model (Q4_0) on CUDA1 with a 45/55 layer split. The benchmark prompt was 994 tokens, not a filled 65K context; generation ranged 41.71–42.45 t/s over 128 output tokens, with 4.5 s average request time and 9.7 s model load.

generation:

30.0 tokens/s

quant:

Q4_K_M (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at 29.96 t/s median completion throughput on an RTX 3090. Setup is llama.cpp (build 10437, CUDA) with GGUF Q4_K_M at 8,192 tokens of context, one concurrent request and no speculative decoding. The model used about 18.0 GB of VRAM on the 24 GB card, which ran at a fixed 280 W power limit. Median latency was 6.94 seconds per question; prompt processing speed was not reported separately.

generation:

35.4 tokens/s

prompt processing (prefill):

355.4 tokens/s

quant:

Q6_K_XL (GGUF)

kv:

f16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User benchmarks Muse Glimmer 30B on AMD v620 GPUs, with Q6 on one GPU at 355.38 t/s prefill and 35.38 t/s generation. Setup is a tensor split across two GPUs: Q6 gives 472.27 t/s prefill and 36.32 t/s generation, Q8 gives 550.04 t/s prefill and 26.55 t/s generation. Speculative decoding uses a dflash draft model.

generation:

173.0 tokens/s

prompt processing (prefill):

2654.0 tokens/s

quant:

UD-Q6_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports speculative decoding with DFlash at a peak of 287 t/s and an average of 173 t/s. Prompt processing runs 1000-2100 t/s on short prompts and 2654 t/s on long ones. Draft acceptance is 63.3-97.5%, with a mean accepted draft length of up to 14 tokens, and KV cache reuse is 99.9%. The user compares it to Qwen 3.6 27B and reports no Chinese language-mixing bugs, no overthinking loops, and concise responses.

generation:

20.0 tokens/s

prompt processing (prefill):

308.0 tokens/s

quant:

UD-Q2-K-XL

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at about 308 t/s prompt processing and about 20 t/s generation on an RX 7600 XT 16GB. Setup is llama.cpp with ROCm, the UD-Q2-K-XL quant and DFlash speculative decoding. The run completed a coding task.

generation:

84.6 tokens/s

quant:

Q5_K_M

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports speculative decoding with DFlash reaching 84.64 t/s at 38.67% acceptance on a best tuned 256K run, against 17.98 t/s for regular decoding. An NVFP4 hybrid reached 94.36 t/s but with worse perplexity, so Q5_K_M was chosen as best. A context fill of 262,116 tokens completed without OOM using 22,920/24,467 MiB VRAM, with decode at 21.56 t/s.

generation:

30.0 tokens/s

quant:

EXL3-SC 3.00bpw H4 (EXL3)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Qwen 3.8 27B at SC2.20bpw H3. User prefers Unsloth UD_Q4_K_XL for coding.

generation:

94.0 tokens/s

prompt processing (prefill):

1400.0 tokens/s

quant:

Q4_K_XL (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Muse Glimmer fits on a single RTX 3090 at Q4_K_XL with full context, DFlash and mmproj, using about 22-23 GB of VRAM. Generation speed ranges from 64 to 124 t/s depending on prose versus code, with an average of 94 t/s. Prompt processing runs at about 1400 t/s. The user notes Qwen3.6-27B and Gemma-4-31B are borderline unusable on the same GPU.

generation:

18.0 tokens/s

quant:

Q4_K_XL (GGUF)

kv:

q4_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports a model fits on a single RTX 5060 Ti 16GB at 131k context with a Q4 KV cache. Setup uses GGUF weights only, with no dflash or mmproj loaded. A Q8 KV cache allows about 90k context.

generation:

233.4 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports a benchmark on an RTX 5090 with 256k context reachable on 24 GB. The model name is Glimmer, which is not in the canonical list and is extracted as raw.

generation:

253.0 tokens/s

quant:

UD-Q5_K_M

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at 253 t/s on an RTX 5090 at 262k context. Setup is llama-server with PR #26842 and DFlash, using a dflash draft model. The run used a code patch workload. The user notes Meta's published figure of 233 t/s.

generation:

90-160 tokens/s

quant:

Q5_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Muse-Glimmer with dflash on an RTX 5090 at roughly 90-160 tok/s depending on the task. The user observes unusual reasoning traces that are disorganized, repetitive, and self-referential, using "we" and mentioning policy. The user compares the reasoning to Gemini models and contrasts it with Qwen 3.6, questions whether reasoning helps or hinders, and also tests a coding prompt for a Flappy Bird HTML game.

generation:

57.0 tokens/s

quant:

BF16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Muse Glimmer 30B at ~57 t/s with speculative decoding using DFlash, against ~25 t/s without speculation, a 2.3x speedup. Setup is vLLM with six patches applied to the image. Mean acceptance length is ~2.5 tokens per verification step, with overall draft acceptance ~10%. Per-position acceptance is ~73% at position 0, ~40% at 1, ~15% at 2, and near zero past 5. The recipe's 3.1x figure was greedy decoding with K-quant on llama.cpp under different conditions. The predicted ceiling is ~65 t/s at 2.6 mean acceptance.