Local model performance on your hardware

generation:

17.6 tokens/s

prompt processing (prefill):

143.2 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Original historical dasLLAMA/Gaijin Entertainment benchmark, measured 2026-09-22. This report is the dasLLAMA arm for Mistral-Small-3.1-24B-Instruct-2503, Q4_K_M. The paired runtime is a separate measured configuration, not a second sample averaged into this headline. Source: https://github.com/GaijinEntertainment/daScript/blob/fa7ce426490e882a60377b99914d626d2c2e2ccb/modules/dasLLAMA/performance/records/m4.json Hardware, captured in this row: Mac16,11, Apple M4 Pro, 14 CPU cores including 10 performance cores, 20-core GPU, 64 GB unified memory, arm64, macOS 26.5.2, remote desktop off. Apple identifies Mac16,11 as Mac mini 2024: https://support.apple.com/en-me/102852 . CPU worker threads 10. Installed unified memory is not dedicated VRAM or model residency; no resident/peak model-memory value or disk capacity is recorded. GPU quantity: 1. The measured commit's metal_common_init has one shared g_dev initialized with metal_create_system_default_device; its C++/Objective-C wrapper calls MTLCreateSystemDefaultDevice. The recorded --ngl 99 selects MetalMode.required, so silent CPU fallback is refused. This binds one participating physical GPU; it does not derive GPU quantity from machine count or core count. Runtime: daslang 0.6.4, dasLLAMA version 16, commit 027907cc8; standalone release executable, tuned on this box. Execution format: weights k4/k6 native; q8 activations; f32 scales; metal blob. Noise=ok and parity=ok are source-recorded gate verdicts, not an independent quality evaluation. Tune sidecar SHA-256 2b2e79baf8ab94bba26d7b1fbfc6576c8b4780293fbef86debe96b26c5d6bdd9. Recorded environment: DASLLAMA_BOX=m4 DASLLAMA_MODELS_DIR=~/models. Native results: PP512 143.17828658159215 tokens/s, sample SD 0.29021206498146057; TG128 17.582416309855425 tokens/s, sample SD 0.11261221766471863. These are actual native processing counts: 512 synthetic prompt positions in PP, 128 serial synthetic decode positions in TG. They are not one 512-input/128-output chat request, context capacity, maximum output budget or achieved visible text length. PP constructs 512 synthetic IDs outside the clock and times one forward_prefill from position 0, including native completion/readback. TG times 128 serial forward calls at positions 0–127, feeding deterministic synthetic IDs; it does not sample text from logits. The native single-step finish waits for its Metal command buffer, and the prefill path waits for its command buffers. Mean and sample SD are over five per-repetition token-count/time rates. Initial model load, tuning, parity/sanity checks and warmup are outside the clock. PP warmup is 512 tokens; TG warmup is one token. Both timings are engine-side loop wall time, not GPU-kernel timestamps or HTTP timing. No independently measured TTFT exists; no 512/PP proxy is promoted to TTFT. Context and conditions: The flat native das session requests min(model supported length, 512 + 128) = 640 positions from the verified runner; this is allocation, not actual input. The source does not publish the final allocator's effective/rounded context. The separate four-stream run sizes a different session. K/V type is f16 in the measured runner. The recorded invocation does not request NextN/MTP speculative decoding or a separate drafter. A carrier filename containing MTP does not make this a speculative text benchmark. Internal native Metal kernel scheduling/encode-ahead is part of the measured implementation; it is not removed or relabeled. The source provides aggregate means/SD only for these cells, so the five individual samples cannot be independently reconstructed. No longer-context performance is inferred. Separate batching evidence: tg128@4 21.135322806679394 tokens/s, sample SD 0.14242206513881683. The separate four-stream value is source-reported summed scheduler throughput, with four synthetic 512-token prompts prefilled outside the generation clock and 128 timed scheduler steps. The timer includes scheduler/event work. It is not single-stream decode, not a 4× multiplication of the headline, and not a claim of a client-serving result. Its pipeline boundary was not independently reconstructed from per-step raw traces. Paired llama.cpp single-stream source values are PP512 146.828459 and TG128 17.366184 tokens/s. Runtime/kernel tuning and allocation differ, so this is a same-carrier comparison with disclosed implementation differences. Weights: upstream unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF, file Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf. Pinned verified artifact: https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/d63ca9416f5db4f54a78145fb9a025317a57289f/Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf . SHA-256 6d670773c3908584349d41a5048d1472226b593c881fd394e8ac196c802e81e2; exact on-disk file length 14333910592 bytes. Both source runtime receipts record that same hash and length. Public HF HEAD metadata matches both, with no weight download. This proves exact tested file contents are available at revision d63ca9416f5db4f54a78145fb9a025317a57289f; it does not prove the original download used that revision. Original HF download revision is unknown. Quantization: Q4_K_M is the source artifact's disk format/recipe, supported by upstream filename, catalog and matching file hash. The record's top-level quant=q8 is the das loader/activation dispatch mode; it does not identify the disk weight format or KV precision. Runtime execution format is preserved above. Source engine metadata records total parameters 23.5724032B and active 23.57198848B; those loader-counted quantities are retained without turning effective-size names into nominal parameter claims. No model-weight bytes are treated as resident RAM. Measurement implementation: https://github.com/GaijinEntertainment/daScript/blob/027907cc8/modules/dasLLAMA/benchmarks/lcpp_bench.das . Device binding: https://github.com/GaijinEntertainment/daScript/blob/027907cc8/modules/dasLLAMA/dasllama/dasllama_metal_common.das . The archived record snapshot was checked byte-for-byte against its pinned public source. CPU-only, audio and image workloads are excluded from this report. LlamaPerf structured rates are rounded to 0.1 tokens/s to match the form precision; the exact source means and SDs are retained above.

generation:

17.4 tokens/s

prompt processing (prefill):

146.8 tokens/s

quant:

Q4_K_M (GGUF)

kv:

F16

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

Original historical dasLLAMA/Gaijin Entertainment benchmark, measured 2026-09-22. This report is the llama.cpp arm for Mistral-Small-3.1-24B-Instruct-2503, Q4_K_M. The paired runtime is a separate measured configuration, not a second sample averaged into this headline. Source: https://github.com/GaijinEntertainment/daScript/blob/fa7ce426490e882a60377b99914d626d2c2e2ccb/modules/dasLLAMA/performance/records/m4.json Hardware, captured in this row: Mac16,11, Apple M4 Pro, 14 CPU cores including 10 performance cores, 20-core GPU, 64 GB unified memory, arm64, macOS 26.5.2, remote desktop off. Apple identifies Mac16,11 as Mac mini 2024: https://support.apple.com/en-me/102852 . CPU worker threads 10. Installed unified memory is not dedicated VRAM or model residency; no resident/peak model-memory value or disk capacity is recorded. GPU quantity: 1. The measured commit's ggml-metal-device.m creates MTLCreateSystemDefaultDevice and explicitly identifies it as the single physical device, physical index 0. Additional GGML_METAL_DEVICES indices, if configured, are virtual devices on that same physical GPU. The recorded backend is Metal with -ngl 99 and no RPC endpoint. This establishes one participating physical GPU without guessing from computer count. Runtime: llama.cpp commit 6fdd0ac89, stock Metal build, 10 CPU threads. The exact binary hash/build number is not published. The recorded native command supplies -ngl 99 -p 512 -n 128 -r 5 -t 10. Source defaults at this commit are batch 2048, microbatch 512, K/V f16, Flash Attention auto, main GPU 0, layer split and automatic load mode. Auto is not rewritten as confirmed Flash Attention on. Native results: PP512 146.828459 tokens/s, sample SD 0.008764; TG128 17.366184 tokens/s, sample SD 0.04137. These are actual native processing counts: 512 synthetic prompt positions in PP, 128 serial synthetic decode positions in TG. They are not one 512-input/128-output chat request, context capacity, maximum output budget or achieved visible text length. PP times 512 synthetic token IDs through llama_decode and llama_synchronize; TG times 128 serial single-token llama_decode calls with synchronization after each. They are separate tests starting from cleared state, with zero pre-existing depth. TG does not sample model answers. Mean and sample SD are the native five-repetition rate statistics retained by the source. Model load and warmup are excluded. PP warmup processes 512 tokens; TG warmup processes one token. Timers are native loop wall time, not HTTP or isolated GPU-kernel clocks. No independently measured TTFT exists; no 512/PP proxy is promoted to TTFT. Context and conditions: The logged commit requests n_ctx=prompt+generation+depth separately for each native test: 512 for PP and 128 for TG at depth 0. Effective allocator-rounded context is not logged; the headline context field remains unknown. The source's separate batched command requests context 4096, batch 2048, microbatch 512, Flash Attention on, four streams each PP512/TG128. The recorded invocation does not request NextN/MTP speculative decoding or a separate drafter. A carrier filename containing MTP does not make this a speculative text benchmark. Internal native Metal kernel scheduling/encode-ahead is part of the measured implementation; it is not removed or relabeled. The source provides aggregate means/SD only for these cells, so the five individual samples cannot be independently reconstructed. No longer-context performance is inferred. Separate batching evidence: tg128@4 17.711999130249023 tokens/s, sample SD 0.004472238477319479. The separate four-stream value is source-reported summed S_TG from llama-batched-bench, not per-user speed or the single-stream headline. Its native decode-call clock differs from dasLLAMA's whole scheduler-step clock, so the batched pair is not asserted to have identical overhead boundaries. It was not reconstructed from individual batched trials here. Paired dasLLAMA single-stream source values are PP512 143.17828658159215 and TG128 17.582416309855425 tokens/s. Runtime/kernel tuning and allocation differ, so this is a same-carrier comparison with disclosed implementation differences. Weights: upstream unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF, file Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf. Pinned verified artifact: https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF/blob/d63ca9416f5db4f54a78145fb9a025317a57289f/Mistral-Small-3.1-24B-Instruct-2503-Q4_K_M.gguf . SHA-256 6d670773c3908584349d41a5048d1472226b593c881fd394e8ac196c802e81e2; exact on-disk file length 14333910592 bytes. Both source runtime receipts record that same hash and length. Public HF HEAD metadata matches both, with no weight download. This proves exact tested file contents are available at revision d63ca9416f5db4f54a78145fb9a025317a57289f; it does not prove the original download used that revision. Original HF download revision is unknown. Quantization: Q4_K_M is the source artifact's disk format/recipe, supported by upstream filename, catalog and matching file hash. The record's top-level quant=q8 is the das loader/activation dispatch mode; it does not identify the disk weight format or KV precision. Runtime execution format is preserved above. Source engine metadata records total parameters 23.5724032B and active 23.57198848B; those loader-counted quantities are retained without turning effective-size names into nominal parameter claims. No model-weight bytes are treated as resident RAM. Measurement implementation: https://github.com/ggml-org/llama.cpp/blob/6fdd0ac89/tools/llama-bench/llama-bench.cpp . Device binding: https://github.com/ggml-org/llama.cpp/blob/6fdd0ac89/ggml/src/ggml-metal/ggml-metal-device.m . The archived record snapshot was checked byte-for-byte against its pinned public source. CPU-only, audio and image workloads are excluded from this report. LlamaPerf structured rates are rounded to 0.1 tokens/s to match the form precision; the exact source means and SDs are retained above.