Nemotron 3.5 Lightning 30B (3B active)

Source: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks Raw measured rows: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/3990a8e0fa5f2762911689439ce12e1c28d4c989/data/speed_2026-10-03.jsonl Quantization/hardware/method README: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/cb99870e05aa9f5783c5c6987ddd9154834123ab/README.md Loaded context metadata: https://github.com/HDZucht/Mac-Mini-M6-LLM-benchmarks/blob/main/data/loaded_context_2026-10-03.jsonl Original author measurements dated October 3–4, 2026, recovered and checked October 13. The raw file has 240 measured rows plus one separate error record. This report includes only the 15 runs for the exact model alias below. No local reproduction was performed. Headline: 45.1 tokens/s is the median of three English-prose runs (Ruhr economy prompt). It is client-observed streaming decode throughput. It is not an overall mean, pooled throughput, peak speed, native kernel benchmark or prefill throughput. Other workloads are kept separate. Hardware: Apple Mac mini M6, Mac18,5, 12 CPU cores (2 Super + 4 Performance + 6 Efficiency), 12 GPU cores, 32 GB unified memory, macOS 27.0.1. One computer; one model loaded alone. Models were stored on an ORICO 2 TB external NVMe with an ASMedia ASM246x enclosure (source reports 40 Gb/s, PCIe 4.0 x4). Fan control was Apple automatic. Per-run temperature, clock, power, swap, peak resident RAM and exact GPU offload are not reported. Measured model alias: lmstudio-community/nvidia-nemotron-3.5-lightning-30b-a3b. Source quantization: GGUF Q4_K_M. Runtime: LM Studio using its llama.cpp GGUF backend; exact application and backend versions/builds are not recorded. The model name/parameter class is source-reported, not an independently counted tensor total. Public availability independently verified: https://huggingface.co/lmstudio-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF/blob/main/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_K_M.gguf. This is a matching publisher/model/quant artifact available on Hugging Face, not proof that this exact current file, revision or hash was used in the original run. The source does not record its GGUF filename, HF revision or hash. Loaded context is verified as 32,768 tokens for this exact alias in loaded_context_2026-10-03.jsonl. This is allocated/loaded capacity, not a filled prompt length. Actual prompt token counts are not retained. Native prompt-processing rate and prefill duration are unknown and the PP field must stay blank. Method: one stream, max_tokens 600, temperature 0.7, reasoning_effort="none", three repetitions for each of five prompts. The script first sends a short Hallo warm-up request, then uses the OpenAI-compatible chat-completions stream. Thinking tokens count if a model emits them anyway. Requested reasoning-off is not independent proof that all model reasoning was disabled. MTP/speculative settings for these ordinary GGUF rows, KV type, batch/microbatch sizes and thread counts are not recorded. The harness computes decode_tps = (tokens - 1) / (stream_end - first_nonempty_content_time). tokens uses API completion_tokens when present, with a nonempty-stream-chunk-count fallback. The saved record includes both tokens and chunks, but not the original usage object, total elapsed time or finish_reason. Values below are the saved, rounded client rates; token counts are preserved as recorded. They are not re-derived from TTFT. TTFT is client wall latency, not native prefill time. Prompts repeat between the three runs; there is no nonce or explicit cache clearing in this script. Prefix-cache state and cached-token counts are unknown, so do not call these cold-prefill tests. Runs at 600 recorded output tokens reached the requested ceiling numerically; finish_reason was not saved. All 15 runs follow. Each run is shown as decode tokens/s; recorded output tokens; nonempty stream chunks; client TTFT seconds. Repetition order is 0, 1, 2. German prose, Rhine geology (wissen_de), median 45.6 tokens/s: Run 0: 45.5 tokens/s; 428 output tokens; 427 chunks; TTFT 0.37 s. Run 1: 45.6 tokens/s; 421 output tokens; 420 chunks; TTFT 0.09 s. Run 2: 45.6 tokens/s; 397 output tokens; 396 chunks; TTFT 0.09 s. German prose, bead-based immunoassay (beads_de), median 45.2 tokens/s: Run 0: 45.2 tokens/s; 586 output tokens; 585 chunks; TTFT 0.48 s. Run 1: 45.4 tokens/s; 600 output tokens; 600 chunks; TTFT 0.12 s. Run 2: 45.2 tokens/s; 566 output tokens; 565 chunks; TTFT 0.31 s. English prose, Ruhr economy (prosa_en), median 45.1 tokens/s: Run 0: 45.2 tokens/s; 402 output tokens; 401 chunks; TTFT 0.49 s. Run 1: 45.1 tokens/s; 391 output tokens; 390 chunks; TTFT 0.13 s. Run 2: 45 tokens/s; 368 output tokens; 367 chunks; TTFT 0.08 s. Python ISO-8601 duration parser (code_py), median 44.9 tokens/s: Run 0: 44.9 tokens/s; 396 output tokens; 395 chunks; TTFT 0.33 s. Run 1: 44.8 tokens/s; 444 output tokens; 443 chunks; TTFT 0.12 s. Run 2: 45 tokens/s; 439 output tokens; 438 chunks; TTFT 0.12 s. JSON antigen/disease extraction (json_extr), median 45 tokens/s: Run 0: 44.1 tokens/s; 55 output tokens; 54 chunks; TTFT 0.36 s. Run 1: 45 tokens/s; 59 output tokens; 58 chunks; TTFT 0.06 s. Run 2: 45.1 tokens/s; 59 output tokens; 58 chunks; TTFT 0.05 s. Quality scope: these tables measure speed. The author reports the JSON task passed all three runs for this model, with automated checking; prose/code quality was not generally graded. Do not infer broader task accuracy from throughput.