Nemotron 3.5 Lightning 30B (3B active)

Source: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks Raw measured rows: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/3990a8e0fa5f2762911689439ce12e1c28d4c989/data/speed_2026-10-03.jsonl Quantization/hardware/method README: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/cb99870e05aa9f5783c5c6987ddd9154834123ab/README.md Loaded context metadata: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/main/data/loaded_context_2026-10-03.jsonl Original author measurements dated October 3–4, 2026, recovered and checked October 13. The raw file has 240 measured rows plus one separate error record. This report includes only the 15 runs for the exact model alias below. No local reproduction was performed. Headline: 42.3 tokens/s is the median of three English-prose runs (Ruhr economy prompt). It is client-observed streaming decode throughput. It is not an overall mean, pooled throughput, peak speed, native kernel benchmark or prefill throughput. Other workloads are kept separate. Hardware: Apple Mac mini M6, Mac18,5, 12 CPU cores (2 Super + 4 Performance + 6 Efficiency), 12 GPU cores, 32 GB unified memory, macOS 27.0.1. One computer; one model loaded alone. Models were stored on an ORICO 2 TB external NVMe with an ASMedia ASM246x enclosure (source reports 40 Gb/s, PCIe 4.0 x4). Fan control was Apple automatic. Per-run temperature, clock, power, swap, peak resident RAM and exact GPU offload are not reported. Measured model alias: unsloth/nvidia-nemotron-3.5-lightning-30b-a3b. Source quantization: GGUF UD-Q3_K_XL. Runtime: LM Studio using its llama.cpp GGUF backend; exact application and backend versions/builds are not recorded. The model name/parameter class is source-reported, not an independently counted tensor total. Public availability independently verified: https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/blob/main/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-UD-Q3_K_XL.gguf. This is a matching publisher/model/quant artifact available on Hugging Face, not proof that this exact current file, revision or hash was used in the original run. The source does not record its GGUF filename, HF revision or hash. Loaded context is verified as 32,768 tokens for this exact alias in loaded_context_2026-10-03.jsonl. This is allocated/loaded capacity, not a filled prompt length. Actual prompt token counts are not retained. Native prompt-processing rate and prefill duration are unknown and the PP field must stay blank. Method: one stream, max_tokens 600, temperature 0.7, reasoning_effort="none", three repetitions for each of five prompts. The script first sends a short Hallo warm-up request, then uses the OpenAI-compatible chat-completions stream. Thinking tokens count if a model emits them anyway. Requested reasoning-off is not independent proof that all model reasoning was disabled. MTP/speculative settings for these ordinary GGUF rows, KV type, batch/microbatch sizes and thread counts are not recorded. The harness computes decode_tps = (tokens - 1) / (stream_end - first_nonempty_content_time). tokens uses API completion_tokens when present, with a nonempty-stream-chunk-count fallback. The saved record includes both tokens and chunks, but not the original usage object, total elapsed time or finish_reason. Values below are the saved, rounded client rates; token counts are preserved as recorded. They are not re-derived from TTFT. TTFT is client wall latency, not native prefill time. Prompts repeat between the three runs; there is no nonce or explicit cache clearing in this script. Prefix-cache state and cached-token counts are unknown, so do not call these cold-prefill tests. Runs at 600 recorded output tokens reached the requested ceiling numerically; finish_reason was not saved. All 15 runs follow. Each run is shown as decode tokens/s; recorded output tokens; nonempty stream chunks; client TTFT seconds. Repetition order is 0, 1, 2. German prose, Rhine geology (wissen_de), median 42.3 tokens/s: Run 0: 42.3 tokens/s; 384 output tokens; 383 chunks; TTFT 0.3 s. Run 1: 42.3 tokens/s; 380 output tokens; 379 chunks; TTFT 0.06 s. Run 2: 42.4 tokens/s; 361 output tokens; 360 chunks; TTFT 0.06 s. German prose, bead-based immunoassay (beads_de), median 42.2 tokens/s: Run 0: 42.2 tokens/s; 562 output tokens; 561 chunks; TTFT 0.22 s. Run 1: 42.1 tokens/s; 556 output tokens; 555 chunks; TTFT 0.06 s. Run 2: 42.3 tokens/s; 558 output tokens; 557 chunks; TTFT 0.06 s. English prose, Ruhr economy (prosa_en), median 42.3 tokens/s: Run 0: 42.1 tokens/s; 384 output tokens; 383 chunks; TTFT 0.2 s. Run 1: 42.4 tokens/s; 359 output tokens; 358 chunks; TTFT 0.06 s. Run 2: 42.3 tokens/s; 373 output tokens; 372 chunks; TTFT 0.06 s. Python ISO-8601 duration parser (code_py), median 42.4 tokens/s: Run 0: 42.4 tokens/s; 470 output tokens; 469 chunks; TTFT 0.23 s. Run 1: 42.4 tokens/s; 474 output tokens; 473 chunks; TTFT 0.06 s. Run 2: 42.4 tokens/s; 365 output tokens; 364 chunks; TTFT 0.06 s. JSON antigen/disease extraction (json_extr), median 42.5 tokens/s: Run 0: 40.7 tokens/s; 52 output tokens; 51 chunks; TTFT 0.24 s. Run 1: 42.5 tokens/s; 68 output tokens; 67 chunks; TTFT 0.05 s. Run 2: 42.7 tokens/s; 55 output tokens; 54 chunks; TTFT 0.05 s. Quality scope: these tables measure speed. The author reports the JSON task passed all three runs for this model, with automated checking; prose/code quality was not generally graded. Do not infer broader task accuracy from throughput.