25 reports. These are individual setups, not a controlled benchmark. Muse speeds by hardware →
M4 Max 128GB · WebGPU
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports running a model in-browser with custom WebGPU kernels, at speed comparable to llama.cpp.
M4 Pro 48GB · mlx-dspark
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports mlx-dspark, a speculative decoding project, running an 8-bit model at 8.2 tok/s baseline and 18-26 tok/s with speculative decoding. Speedups are 3.27x on math, 2.5x on code and 2.22x on chat. The 4-bit model reaches about 1.7x at about 25 tok/s and needs about 18 GB. The 8-bit model peaks at about 40 GB and requires a 48 GB Mac. Output is byte-identical to normal decoding.
NVIDIA RTX Pro 6000 Blackwell · TensorSharp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User benchmarks TensorSharp against llama.cpp on an RTX PRO 6000 Blackwell, with plain text generation decode at about 35 tok/s at 60 prompt tokens. The user also tests DFlash speculative decoding and 2x RTX PRO 4000 Blackwell 24GB with tensor parallelism.
NVIDIA RTX 5090 · vLLM · 25,000 ctx
User reports 421 tok/s at 25k context on an RTX 5090 with muse-glimmer 30B, using a v2.2 optimizer profile for vLLM. Setup runs without DFlash speculative decoding, with TTFT at 224ms (p95 500ms) and TPOT at 23ms. Cost drops from $3.41 to $0.65 per 1M output tokens and energy from 4.72 to 1.08 J/tok. The user mentions SWE-Bench agents.
2× NVIDIA DGX Spark · llama.cpp · 1,000,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports retrieval at 832K tokens with 3/3 success. DFlash speculative decoding gives roughly 3x speedup. RPC split is slower than single-node.