Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports approximately 15 tokens/s generation speed running Qwen3.8-27B on a DGX Spark (GB10) with llama.cpp.
Setup is llama.cpp with UD-Q4_K_XL GGUF quant and q8_0 KV cache, using MTP speculative decoding with draft-mtp and spec-draft-n-max 2, batch 4096, ubatch 2048, flash attention on, 99 GPU layers, 8 threads.
User notes that NVIDIA forum users reported similar performance with vLLM, but an official SGLang post reported 38 tokens/s decode throughput on DGX Spark, and asks whether a configuration or optimization is missing or whether the model/platform is not yet fully optimized in llama.cpp.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-use
User reports Qwen3.8-27B at 47.5 tok/s on a DGX Spark (GB10).
Setup is the exllamav3 fork with EXL3 3.5bpw weights, nvfp4 KV cache, and DFlash2 speculative decoding (EXL3 5.0bpw draft, 1.4 GB).
The 47.5 tok/s figure is HumanEval-class decode at T=0.6 with acceptance 4.43 tokens/step; the kit also cites ~30 tok/s with MTP and ~34.5 tok/s with DFlash2 at 24 GB, and ~16.7 tok/s aggregate across 8 concurrent requests.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports GLM 5.3 Flash at around 22 tok/s on a dual DGX Spark setup.
Setup is two DGX Sparks out of a four-Spark cluster.
User found GLM 5.3 overly verbose and slow, and is now running DeepSeek V4 0731 on two Sparks and Qwen 3.8 Flash on the other two.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports MiMo V2.6 Pro RL at 68.3 t/s on eight DGX Spark units, up from 17.8 t/s without speculation.
Setup is vLLM with DFlash speculative decoding on official weights; the figure includes prefill and request overhead.
Outputs matched byte for byte across 12 cases, including two baseline mistakes. Four concurrent requests and a 257K-token input were also tested.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agentictool-usecodinglong-context
User reports MiMo-V2.6-Flash-RL at ~42–51 t/s on 2× DGX Spark (GB10) with vLLM, TP=2 over RoCE, fp8 KV cache, full 1,048,576 context, and DFlash with 7 draft tokens.
Setup uses the tonyd2wild recipe's sm121-v11-dflash2 image. Decode is ~42–51 t/s on agentic code at default sampling, 69 t/s on the recipe's coding benchmark at temperature 0, and ~20–23 t/s on prose. Deep prefill is the weak spot at ~160 t/s past 800K, so a cold 1M prompt takes ~67 min.
User documents three serving bugs: empty streaming replies with thinking on (fixed by pre-opening the thinking tag in the chat template), dropped earlier reasoning in tool loops (fixed by template and chat_utils.py fallbacks), and a hidden 2,048-token output cap from generation_config.json (fixed with --override-generation-config max_new_tokens 131072). A 1M needle test retrieved 3/3 hidden codes from a 996K-token prompt.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagenticlong-context
User reports running Kimi K3 (2.8T) on a 16x GB10 cluster at ~30 t/s generation (peaking ~38 t/s) for coding and agentic tasks.
Setup uses a customized gb10-vllm stack with custom MLA/KV kernels, dual MikroTik switches, and 4x 400G-to-4x100G breakout cables. Prefill throughput is ~750–910 t/s.
Stable multi-hundred-thousand token context runs with multiple 500k compaction; handles concurrent requests without dropping token generation rates.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8 Flash-Next at about 35 t/s on a single DGX Spark.
Setup is SGLang with an NVFP4 quant at 262,144 context, used through VSCode Copilot in autopilot mode.
The session produced about 10k lines of code and consumed about 800k tokens over 8 hours of planning, coding, and testing.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-usevisionmultilingual
User reports Qwen3.8-Flash-Next (180B MoE, 7.31B active) at 43.9 t/s on a single DGX Spark, roughly four times the 11.2 t/s of Qwen3.8-27B dense on the same machine.
Setup is vLLM with NVFP4 weights, FP8 KV cache, MTP depth 3, greedy sampling, thinking mode off, tensor parallel size 1, and 320 tokens written per test. The 123.53 GiB checkpoint fits in 121.7 GiB unified memory because the 51.2B-parameter PLE n-gram table (47.68 GiB) is read from NVMe per token while the 120.8B experts (56.25 GiB at NVFP4) stay resident.
Tool calls, thinking mode, and image input work on the live server; video input was accepted but the test clip was faulty so that result is inconclusive.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
long-context
User compares generation throughput on a 128 GB DGX Spark at three starting-context depths for two deployment paths. Official INT4 on a vLLM fork reaches 38.3 t/s short, 7.9 t/s at ~45K and 4.6 t/s at ~90K. Q5_K_M on llama.cpp reaches 35.7 t/s short, 33.6 t/s at ~45K and 33.2 t/s at ~90K.
The config is 262,144 tokens of max context with ~103 GB total memory used, and streaming separates TTFT from decode.
These are the creator's measurements, not an independent rerun, and two variables change at once (runtime and quantization), so no pure engine effect is isolated. The later 131,072-context serving default should not be attached to this table.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User benchmarks Ling-3.0-flash on a DGX Spark at 40.9 t/s on a short coding task.
Setup is CUDA graphs with MTP n=1.
Prose throughput is 38.7 t/s at 512-token output and 37.3 t/s at 2048-token output. MTP n=2 and n=3 are slower for prose. Acceptance lengths are 1.87 at n=1, 2.39 at n=2 and 2.77 at n=3. Without MTP the baseline is 22.9 t/s with CUDA graphs and 20.8 t/s eager.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash on two ASUS GX10 (DGX Spark) units connected via RoCE with TP=2, decoding 49.4 t/s at 4K context, 43.0 t/s at 16K, 37.9 t/s at 32K, 42.5 t/s at 128K and 39.8 t/s at 256K, with prefill of 2050 t/s, 2150 t/s, 2130 t/s, 1920 t/s and 1680 t/s at the same context lengths.
Setup is a vLLM fork from local-inference-lab with MXFP8 x MXFP4 weights.
At concurrency 4 and 128K context the aggregate is about 40-42 t/s. The KV cache holds up to about 1M tokens of context, though the user typically runs 256K. The user reports roughly 280W at maximum load and says the model beats M2.7 and Stepfun 3.7 on a private benchmark, praising its high-context retrieval and reasoning. A Docker compose file is provided.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash at about 40 t/s single-stream and 350 t/s aggregate with 32 concurrent requests at 256k context on dual DGX Sparks.
The 350 t/s figure is an aggregate across the 32 concurrent requests.
User compares this to an RTX Pro 6000 at about 46 t/s and a Mac M2 Ultra 192GB at about 29 t/s, both at Q2.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentic
User reports DeepSeek V4 Flash at 1800 t/s prefill and 40 t/s generation on two DGX Spark units.
User praises the scalability via ConnectX and the power efficiency.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
long-contextcoding
User reports GLM-5.2 NVFP4 at about 15 t/s decode on short context and about 13 t/s at long context on a 4x DGX Spark setup, with about 512 t/s prefill.
Setup is 128K context with TP4/PP1/DCP4/MTP1 and an fp8 KV cache.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports REAP-pruned DeepSeek-V4-Flash served at 262k context on a single DGX Spark (Ascent GX10).
Setup is vLLM.
Prefill and generation throughput remain stable from 4K to 162K context. Benchmarks include pp and tg at various context lengths and concurrency levels.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 at 155.43 GiB FP8 across 48 shards on a DGX Spark (GB10, aarch64, sm_121, 121.7 GiB unified memory, 128 GiB swapfile), with prefill steady at 1000 tps and decode at about 19 tok/s.
Setup is vLLM-Moet (vLLM v0.25.0 plus patch) with 2-bit MoE experts, a KV cache of 4.56M tokens at 512K context and util 0.90, and the MTP head from ycui7/DeepSeek-V4-Flash-MTP.
With the MTP head decode reaches 26.6 tok/s, 48% over plain. Aggregate MTP versus plain is 25.2 versus 19.1 at conc1, 30.9 versus 26.4 at conc2, and 43.2 versus 45.6 at conc4. Per-request MTP versus plain is 25.2 versus 19.1 at conc1, 23.0 versus 17.7 at conc2, and 14.5 versus 13.9 at conc4. The 2-bit planes are 43 layers x 1.69 GiB, about 73 GiB. Boot takes about 10 min warm and 31-46 min cold.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports the MXFP4 version from Bartoswski on a DGX Spark, with prefill at 1K t/s and generation averaging 90 t/s.
User calls it very efficient and the best scoring yet, and notes it lacks vision.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports DeepSeek-V4-Flash-0731 UD-IQ3_XXS at roughly 23-27 t/s on a DGX Spark GB10, stabilizing at about 24 t/s.
Setup is llama.cpp with DSpark speculative decoding.
The user notes it is faster than Qwen3.6-27B-NVFP4 for codegen.
User reports a C++20 port of the vLLM serving stack on a DGX Spark with Qwen3.6-27B NVFP4, reaching 86.05 to 1095.01 output tokens/sec across concurrency 1 to 32.
The port runs slightly ahead of vLLM in the same tests.
The user also reports DeepSeek-V4-Flash in 2-bit GGUF at 18.69 tok/s on the Spark, and compares against llama.cpp and MLX-LM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek-V4-Flash-0731 (304B MoE) at 82 tok/s decode and about 1400 tok/s prefill on 2x DGX Spark, each with 128GB unified memory.
Setup is vLLM 0.26.1rc1 with TP=2, 1M context and FP8.
User asks for advice on reducing VRAM and OS RAM usage.