- generation:
- 66.8 tokens/s
- prompt processing (prefill):
- 1450.3 tokens/s
- quant:
- Q4_0 (GGUF)
- kv:
- F16
- flash attention:
- on
- mtp (multi-token prediction):
- off
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
Source: https://github.com/hogeheer499-commits/strix-halo-guide/blob/f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234/data/raw/2026-09-26/64gb-tier-30b-candidates-b11146/README.md Author: hogeheer499-commits. Historical measurement 2026-09-26, recovered 2026-10-13; source snapshot f5d9e6dac1ec9aeeeeb1165ae85d305b33cee234. Raw CSV and JSON, per-run commands and stderr retained. This is routine-load first-party benchmark evidence, not independent reproduction. Hardware: Beelink GTR9 Pro, AMD Ryzen AI Max+ 395 / Radeon 8060S, nominal 128GB unified RAM. Each of the four measured processes explicitly enumerates exactly one Vulkan device, Radeon 8060S; native rows say Vulkan. RPC library being loaded is not RPC participation. Native CPU threads16. Separate May1 hardware audit says16 physical cores/32 threads; these are historical same-host topology, not an active-core measurement in September. GPU CU count unknown. No dedicated VRAM or resident/peak model memory measurement. Runtime: official Ubuntu x64 Vulkan llama.cpp v0.5.0 release asset, executable reports0.5.0-dev build11146 commit7fe450e19, GNU11.4.0. Kernel7.0.0-31, Mesa/RADV26.2.3, AMD_VULKAN_ICD=RADV. All rows use ngl999, FA1, batch2048, ubatch512, F16 K/V, load/lazy auto, devices:auto, main_gpu0, layer split, no CPU MoE override. Host: balanced power, DPMauto, VM/desktop/browser/video-call/remote-access processes active; Ollama empty; preexisting swap nearly full. No strict-clean or causal cross-campaign comparison. Model: NVIDIA Nemotron 3.5 Lightning 30B-A3B, Q4_0; publisher ggml-org; NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. GGUF native type is nemotron_h_moe 31B.A3.5B Q4_0; metadata parameter count 31.577940288B differs from the rounded marketed 30B-A3B name. No MTP sidecar or speculative decoding. Artifact: pinned HF 8a08a1c81dadcc75d35dbb96016cfd344b632e67; 18898091584 disk bytes; SHA256 61f87e75974e4b535dcdf9aad056541a9514f1dfa4538b463b081d19b7a00e3c. Source's local hash and byte count match independently retrieved pinned HF metadata. Actual tested bytes remain author-attested; no weight download or independent rehash. Native model_size=18890195712 is tensor storage, not disk file size or residency. HF: https://huggingface.co/ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/blob/8a08a1c81dadcc75d35dbb96016cfd344b632e67/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf Native rows (mean +/- sample SD; pass2 is a distinct repeated control, not pooled or best-of): - Pass 1 pp512, prepared depth 0, r10: 1450.331991 +/- 3.306742 t/s; mean native duration 353024272 ns. - Pass 1 tg128, prepared depth 0, r10: 66.801892 +/- 0.140252 t/s; mean native duration 1916120989 ns. - Pass 2 pp512, prepared depth 0, r10: 1446.360644 +/- 6.329379 t/s; mean native duration 353998024 ns. - Pass 2 tg128, prepared depth 0, r10: 67.147905 +/- 0.082188 t/s; mean native duration 1906242227 ns. - Pass 1 pp8192, prepared depth 0, r3: 1409.099000 +/- 2.766984 t/s; mean native duration 5813658993 ns. - Pass 1 tg128, prepared depth 8192, r3: 65.987632 +/- 0.078190 t/s; mean native duration 1939759257 ns. The headline pair is pass1 PP512/TG128. Separate synthetic tests, not a512-input/128-output chat request. Exact commit's llama-bench timer inspected: load, context creation, warmup and depth preparation precede timing; PP processes random token batches and synchronizes; TG does128 single-token decode calls with synchronization. Numerator is n_prompt+n_gen, not useful natural-language output. The depth8192 TG test preloads/restores8192 tokens outside timing, then measures128 new decode steps. PP8192 starts at depth0. Requested n_ctx is512/128/8192/8320 per corresponding shape; effective allocated capacity is not logged and stays unknown. Native durations are not TTFT. Measured TTFT unknown; no reciprocal-rate or prompt/time proxy supplied. No server/client rate, concurrency, speculation or quality inference imported. Timer: https://github.com/ggml-org/llama.cpp/blob/7fe450e19/tools/llama-bench/llama-bench.cpp LlamaPerf numeric fields are rounded to the supported 0.1 tok/s step; exact source means and standard deviations are retained above.