Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcoding
User reports quantizing ukisai's Swift 1.5 Qwen3.8-Flash-Next to NVFP4 for a 2x DGX Spark pair running stock dgpp-serve with TP=2. The 126 GiB checkpoint runs at ~55-60 tok/s C1 and ~114 tok/s C4 on prose, with prefill ~1.9-2.3k tok/s.
Setup is dgpp with NVFP4 weights, 126 GiB, 296k tensors, 52 shards, no engine changes.
The fine-tune is a reasoning-efficiency variant that cut output ~24% (805 to 611 tokens) on a chat probe with no speed loss, but savings are prompt-dependent and one zebra puzzle went up 39%; the upstream 31-57% thinking-token claim was not reproduced on a real eval suite.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Gemma 4 31B at about 6.0 t/s on a DGX Spark without MTP, and 11.4 to 19.3 t/s with MTP speculative decoding enabled (--spec-draft-n-max 4).
Setup is llama.cpp with the am17an/Gemma4-31B-it-GGUF build.
With MTP the draft accept rate was 0.5881 over 9 requests.
The user notes no speed-up for the MoE variant, over 2x average speedup on the dense model, and that AIME-26 results (~87%) were replicated.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
mathtool-usemultilingualvisionlong-context
User reports Gemma 4 26B-A4B-it NVFP4 at 48.2 t/s for a 1000-token generation on an NVIDIA DGX Spark (GB10 Blackwell, 128GB unified memory), against 23.3 t/s for the BF16 original.
Setup is vLLM with an NVFP4 (W4A4) modelopt checkpoint and FP8 KV cache, single DGX Spark, Marlin MoE backend, 256K max context; the model loads at 15.7 GiB.
TTFT is 53 ms versus 97 ms for BF16, and the checkpoint is ~16.5 GB on disk versus ~49 GB. Quality retention averages 97.6% across GSM8K and IFEval, with GSM8K down about 4 percentage points.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 Flash-Next at 39.8 t/s mean decode on two DGX Sparks, with final code decode at roughly 74 t/s mean.
Setup is vLLM with FP8 weights and NVFP4 KV cache, TP=2 across two DGX Sparks, DSpark speculative decoding with 3 proposed tokens, single concurrency.
User also tried DeepSeek V4 Flash on the same setup but it failed benchmarks due to non-completion and excessive token use (30-45k tokens on an easy task), unlike cloud versions of the same model. User is happy with Qwen3.8 Flash-Next but concerned about overthinking tendency.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4.1-Flash at 37.95 tok/s single-stream decode across 8 categories and 902 tok/s cold prefill at 2K tokens on four DGX Sparks.
Setup is vLLM TP4 across four DGX Sparks over ConnectX-7 RoCE, MXFP4 experts and MXFP8 dense weights, DSpark speculative decoding with k=5, the whole model in unified memory (56.6-68.9 GiB per Spark), 300K context window.
Per category the single-stream decode was code 73.8 tok/s, JSON 52.1, math 50.9 and prose 24.4. Six streams reached 131.86 tok/s aggregate. Cold prefill was 1,539 tok/s at 32K and 1,194 at 64K.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.6 27B at 7-11 tok/s with stock llama.cpp on an NVIDIA DGX Spark (GB10), consistent across every scenario tested.
Setup is stock llama.cpp on one GB10 with no speculative decoding and a q8_0 KV cache, the whole model in unified memory; the post does not state the target quant for the stock run.
With DFlash speculative decoding (spiritbuun fork, Q4_K_M target, Q8_0 draft, turbo4 KV) the author gives per-task ranges only: 38-40 tok/s HTML/JS, 24-25 Python, 23-25 short chat, 20-22 medium context and 27-29 sustained over 2048 tokens. Q8_0 and BF16 targets are slower. vLLM with MTP reached around 30 tok/s. The 35B-A3B MoE runs 60-66 tok/s stock and up to 113 tok/s on HTML/JS with DFlash.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticvision
User reports GLM-5.3 Flash NVFP4 on two DGX Sparks at a 66.0 t/s mean decode across a task battery, with prefill at 1376 t/s on a 1.5K prompt.
Setup is the GLM-5.3-Flash-NVFP4-DFlash2 recipe with DFlash2 speculative decoding, 3.55 of 8 draft tokens accepted per step, single request, and a KV pool of 560,362 tokens.
Battery tasks range from 37.1 t/s on prose to 97.2 on bulk SQL inserts, and prefill holds near 1,576 t/s at 32K and 128K. The previous DeepSeek V4 Flash setup averaged 61.5 t/s on the same battery. A 40-minute soak at c4 served 503 requests with 0 errors at 83.6 t/s aggregate.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-contexttool-use
User reports DeepSeek V4 Flash 162B (REAP K144 pruned) at 13.899 tok/s decode and 538.958 tok/s prefill on a single DGX Spark.
Setup is vLLM with NVFP4/MXFP4 expert weights and FP8 KV cache, 200K context, 14 GB KV cache, no speculative decoding.
Validation used a 186,390-token prompt with 345.834 s TTFT and needle retained; the watchdog logged a low-memory kill at teardown, so the profile is proof-of-concept. K160 prefills at about 514 tok/s and decodes at about 24 tok/s with MTP.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentictool-uselong-context
User reports DeepSeek-V4-Flash-0731 at ~9.8 tok/s decode on a single DGX Spark (GB10, 128GB unified memory).
Setup is a vLLM-based stack (vLLM-Moet adapted for unified memory and sm_121) with 2-bit expert planes and an FP4 tier, serving 256K context with a KV pool of ~660K tokens.
Decode uses the built-in DSpark speculative head at k=2; the recommended k=7 was slower at 5.1 tok/s. Prefill is ~500 tok/s chunked at 2048, with a 28K-token prompt giving ~53 s TTFT. First boot took ~35 min for one-time expert quantization, later boots ~10 min via persistent cache. Tool calling and per-request thinking toggle work out of the box.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
visioncodingagenticlong-context
User reports DeepSeek-V4-Flash-Vision-Exp at 80.1 tok/s on count-to-100, 51.8 tok/s on code and 33.2 tok/s on prose, on 2x DGX Spark at TP2.
Setup is vLLM with NVFP4 weights and nvfp4_ds_mla KV cache, 1M context, DSpark speculative decoding at k=5, KV pool 2.79M tokens / 19.12 GiB at gpu_memory_utilization 0.85.
Without the Patch 4 shared-expert fix the same workloads measure 50.7, 47.2 and 30.4 tok/s. The 0731 text checkpoint reaches 78.4 tok/s peak at 98.9% acceptance with Patch 4, versus 66.1 peak and 60.2% acceptance without. The user notes a ~30% cold-start penalty and that realistic mixed agent traffic at concurrency 4 gives ~88.6 tok/s aggregate and ~22.3 tok/s per stream.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-contextagenticcoding
User reports DeepSeek-V4-Flash at ~41 tok/s single-stream and ~350 tok/s aggregate at c=32 on 2x DGX Spark (GB10, 128 GB unified) via vLLM TP=2 over QSFP56 RoCE.
Setup is vLLM with official FP8 weights, 1M context profile, shared ~1.1M-token KV pool; prefill ~1785 t/s.
Long-context sweep: 8k 40 t/s, 100k 40 t/s, 200k 38 t/s, 500k 32 t/s decode; prefill ~1970 down to ~1380 t/s. Cross-machine: RTX PRO 6000 ds4 46.9 t/s, M2 Ultra 29.7 t/s, single Spark ds4 IQ2_XXS ~14 t/s. llama.cpp GGUF suite up to 67 t/s (35B-A3B), 172B MoE 31 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports DeepSeek V4 Flash 0731 at 44-47 tok/s decode on one NVIDIA DGX Spark (GB10, 128 GiB unified memory).
Setup is vLLM with EXL3 3.0 bpw weights, NVFP4 KV cache, 384k context, DSpark K5 speculative decoding with a K64 draft model, single sequence.
The 44-47 tok/s is a range. A 370,104-token stress test achieved exact needle recall with 0 preemptions, prefill ~625 tok/s effective, 594 s end-to-end. KV pool is 439,622 tokens.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-usevisionlong-context
User reports DeepSeek-V4.1-Flash at 87.7 tok/s single-stream decode on four NVIDIA DGX Sparks (TP=4).
Setup is SGLang with native MXFP4 experts and FP8 dense weights, DSpark speculative decoding (k=5), 1M context configured, 8M-token KV pool pinned, ~77 GiB resident weights per rank.
Aggregate decode reaches 342.7 tok/s at 16 streams (22.2 per stream); cold prefill is 4,059-5,925 tok/s at 4k-262k context, 4.9k-5.0k tok/s on real text at 16k-128k. A 1,011,084-token needle test passed. The 3-Spark TP=3 profile is also documented at 37.9 tok/s single-stream.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
visioncodinglong-contextagentic
User reports a two-node DGX Spark recipe for DeepSeek V4 Flash Vision-Exp under vLLM with TP=2 and DSpark speculative decoding, giving ~62-83 decode tok/s on a single chat through 128K context.
Setup is vLLM with NVFP4 weights and nvfp4_ds_mla KV cache, 17.04 GiB KV pool holding 2,331,430 tokens at 0.835 GPU utilization, 1M-token ceiling and 6 concurrent sequences.
Six short chats reach ~160-190 tok/s aggregate (~30-37 per stream); a 2026-08-14 run measured 162 aggregate at c=6 and 75 tok/s at 128K single-stream, while a 2026-09-02 run measured 56 tok/s at 256 tokens and 139 aggregate at c=6. A three-Spark TP=3 lane gives ~200 tok/s aggregate at 16 streams with 4-13% faster decode per stream but 4-13% slower prefill to 64K and ~22% slower at 128K-256K. The 315/205 tok/s figures need the Stage-C plus Keys path.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagenticlong-contextsummarization
User benchmarks Qwen3.8-Flash-Next on a DGX Spark (GB10, 121 GB unified memory) against Claude Opus 5.5 across 21 graded coding tasks. Local model matches Opus on 11 everyday tasks (11/11 each) but is roughly 5-8x slower, and scores 7/10 on hard tasks versus Opus's 10/10.
Setup is vLLM 0.30 with NVFP4 weights and MTP speculative decoding, about 30 tok/s on long generations. Harnesses tested were a minimal 3-tool loop, Claude Code 2.1.285, and Qwen Code 0.24.7.
User notes the failure mode was over-verification rather than inability, and that harness choice moved the hard-tier score as much as settings did. A real-world test built a 150-line spec wedding microsite in 88 minutes, about 5.4k lines across 30 files.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcodingmathlong-context
User reports DeepSeek V4.1 Flash at 43.0 tok/s speculative decode on a single NVIDIA DGX Spark (GB10, 128 GB unified memory), using a custom YoungAi C/CUDA engine.
Setup is a 113.6 GB VQ-8 base GGUF plus a ~40 MB per-domain sidecar; the 14.1k-token Agent request ran greedy with speculative decoding at 3.04 tok/round. Pure decode on the same request is 28.9-29.4 tok/s, and short-prompt greedy is 30.5-30.7 tok/s.
Prefill is 1,055 tok/s on a 12.5k prompt, 940 tok/s on the 14.1k Agent request, and 671 tok/s on a 106.7k prompt (159 s TTFT). Domain sidecars raise top-1 agreement vs the original by +2.8 to +3.7 points across finance, code, law, medicine and science; the post-training file flips decisions on the solving request but does not generalize to held-out days.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-usevisionlong-contextsummarization
User reports Ornith-1.5-35B-A3B at 79 tok/s single-stream on one NVIDIA DGX Spark (128GB unified memory).
Setup is vLLM 0.24 with the official 4-bit NVFP4 checkpoint (about 22 GiB) and multi-token prediction speculative decoding, serving the full 262K context.
At 20 concurrent users each stream gets about 25 tok/s, 490 tok/s total; GPU memory utilization had to be capped at 0.70 and throughput dropped past 20 users. Reasoning scores cited: MATH-500 97%, GSM8K 98%, MMLU-Pro 81%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports DeepSeek V4 Flash 0731 running at 60-70 t/s on dual DGX Spark (ASUS GX10) with native quant.
Setup uses two DGX Spark units for agent work, including porting a ~170k LOC Node codebase to Rust.
The user asks what others use for coordinating multiple agents with local models.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentictool-use
User reports Swift 1.5 (a fine-tune of Qwen3.8-Flash-Next) at ~110.2 tok/s single-stream decode on two DGX Sparks with veloGB10, TP=2.
Setup is veloGB10 with an EXL3 4.05 pack (SharkWipf) and MTP speculative decoding; the official doth4580 pack reached 110.6 tok/s and a Huihui abliterated pack 113.6 tok/s on the same hardware. A header-only rewrite of the n-gram shard layout was needed to load community EXL3 packs.
User compares against an older vLLM NVFP4 setup at ~52 tok/s (September, not same-day) and reports Swift 1.5 at xhigh effort passing 11/12 hard trap tasks versus 7/12 at medium, at about 2x output tokens and 1.8x wall time. Aggregate throughput at 2-3 concurrent requests is 91-98 tok/s, rising to 157 at 8 and 181 at 16.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingvisionagentic
User reports Ornith-1.5-35B-A3B APEX i-quality at 69.0 t/s on a DGX Spark (GB10).
Setup is llama.cpp with Q6_K/Q5_K/Q3_K banded GGUF at 131072 context, single slot, no speculation. A second no-drafter run measured 69.9 t/s.
With MTP draft-mtp at n_max 1 the model reached 73.52 t/s (0.674 acceptance); deeper draft depths were slower. i-compact measured 73.9 t/s and i-mini 80.8 t/s via llama-cli. The agentic coding suite passed 9 of 9 tasks and 154 of 154 tests; vision passed 5 of 5 with the image and 0 of 5 blind.