- generation:
- 67.5 tokens/s
- prompt processing (prefill):
- 1158.7 tokens/s
- quant:
- Q8_0 (GGUF)
- kv:
- F16
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
Original historical dasLLAMA/Gaijin Entertainment benchmark, measured 2026-09-23. This report is the llama.cpp arm for Qwen2.5-Omni-3B, Q8_0. The paired runtime is a separate measured configuration, not a second sample averaged into this headline. Source: https://github.com/GaijinEntertainment/daScript/blob/fa7ce426490e882a60377b99914d626d2c2e2ccb/modules/dasLLAMA/performance/records/m4.json Hardware, captured in this row: Mac16,11, Apple M4 Pro, 14 CPU cores including 10 performance cores, 20-core GPU, 64 GB unified memory, arm64, macOS 26.5.2, remote desktop off. Apple identifies Mac16,11 as Mac mini 2024: https://support.apple.com/en-me/102852 . CPU worker threads 10. Installed unified memory is not dedicated VRAM or model residency; no resident/peak model-memory value or disk capacity is recorded. GPU quantity: 1. The measured commit's ggml-metal-device.m creates MTLCreateSystemDefaultDevice and explicitly identifies it as the single physical device, physical index 0. Additional GGML_METAL_DEVICES indices, if configured, are virtual devices on that same physical GPU. The recorded backend is Metal with -ngl 99 and no RPC endpoint. This establishes one participating physical GPU without guessing from computer count. Runtime: llama.cpp commit 6fdd0ac89, stock Metal build, 10 CPU threads. The exact binary hash/build number is not published. The recorded native command supplies -ngl 99 -p 512 -n 128 -r 5 -t 10. Source defaults at this commit are batch 2048, microbatch 512, K/V f16, Flash Attention auto, main GPU 0, layer split and automatic load mode. Auto is not rewritten as confirmed Flash Attention on. Native results: PP512 1158.744138 tokens/s, sample SD 0.309465; TG128 67.501254 tokens/s, sample SD 0.041464. These are actual native processing counts: 512 synthetic prompt positions in PP, 128 serial synthetic decode positions in TG. They are not one 512-input/128-output chat request, context capacity, maximum output budget or achieved visible text length. PP times 512 synthetic token IDs through llama_decode and llama_synchronize; TG times 128 serial single-token llama_decode calls with synchronization after each. They are separate tests starting from cleared state, with zero pre-existing depth. TG does not sample model answers. Mean and sample SD are the native five-repetition rate statistics retained by the source. Model load and warmup are excluded. PP warmup processes 512 tokens; TG warmup processes one token. Timers are native loop wall time, not HTTP or isolated GPU-kernel clocks. No independently measured TTFT exists; no 512/PP proxy is promoted to TTFT. Context and conditions: The logged commit requests n_ctx=prompt+generation+depth separately for each native test: 512 for PP and 128 for TG at depth 0. Effective allocator-rounded context is not logged; the headline context field remains unknown. The source's separate batched command requests context 4096, batch 2048, microbatch 512, Flash Attention on, four streams each PP512/TG128. The recorded invocation does not request NextN/MTP speculative decoding or a separate drafter. A carrier filename containing MTP does not make this a speculative text benchmark. Internal native Metal kernel scheduling/encode-ahead is part of the measured implementation; it is not removed or relabeled. The source provides aggregate means/SD only for these cells, so the five individual samples cannot be independently reconstructed. No longer-context performance is inferred. Separate batching evidence: tg128@4 146.08199768066407 tokens/s, sample SD 0.1430727243423462. The separate four-stream value is source-reported summed S_TG from llama-batched-bench, not per-user speed or the single-stream headline. Its native decode-call clock differs from dasLLAMA's whole scheduler-step clock, so the batched pair is not asserted to have identical overhead boundaries. It was not reconstructed from individual batched trials here. Paired dasLLAMA single-stream source values are PP512 1245.559692075903 and TG128 70.90394559256633 tokens/s. Runtime/kernel tuning and allocation differ, so this is a same-carrier comparison with disclosed implementation differences. Weights: upstream ggml-org/Qwen2.5-Omni-3B-GGUF, file Qwen2.5-Omni-3B-Q8_0.gguf. Pinned verified artifact: https://huggingface.co/ggml-org/Qwen2.5-Omni-3B-GGUF/blob/75f1b73b657a50f5092502799457ccb4a4a1f9df/Qwen2.5-Omni-3B-Q8_0.gguf . SHA-256 c7d8b07c8d8d7a9ed1de1b8df7ac821eb4d259a224bd44310baacfaa5a473d4c; exact on-disk file length 3616087360 bytes. Both source runtime receipts record that same hash and length. Public HF HEAD metadata matches both, with no weight download. This proves exact tested file contents are available at revision 75f1b73b657a50f5092502799457ccb4a4a1f9df; it does not prove the original download used that revision. Original HF download revision is unknown. Quantization: Q8_0 is the source artifact's disk format/recipe, supported by upstream filename, catalog and matching file hash. The record's top-level quant=q8 is the das loader/activation dispatch mode; it does not identify the disk weight format or KV precision. Runtime execution format is preserved above. Source engine metadata records total parameters 3.397103616B and active 3.396861952B; those loader-counted quantities are retained without turning effective-size names into nominal parameter claims. No model-weight bytes are treated as resident RAM. Measurement implementation: https://github.com/ggml-org/llama.cpp/blob/6fdd0ac89/tools/llama-bench/llama-bench.cpp . Device binding: https://github.com/ggml-org/llama.cpp/blob/6fdd0ac89/ggml/src/ggml-metal/ggml-metal-device.m . The archived record snapshot was checked byte-for-byte against its pinned public source. CPU-only, audio and image workloads are excluded from this report. This is exclusively a synthetic text-token workload of the language model; no image, audio, projection file or multimodal encoder was timed. It is not a vision/audio performance result. LlamaPerf structured rates are rounded to 0.1 tokens/s to match the form precision; the exact source means and SDs are retained above.