DeepSeek V4 Flash

reported speed:

60.0 tokens/s generation

quant:

FP4

kv:

FP8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

10-12x RTX 3090, 285B MoE, FP4 experts + FP8 attention, 157GB weights. 60+ tok/s on 10 GPUs (TP2xPP5) at 240W cap, 120+ tok/s on 12 GPUs (TP4xPP3). 1M context no offload, 4M with RAM offload. ~3500 tok/s long-context prefill (batched). Vision + spec + tool calls working. Docker image and repo provided.

reported speed:

5.5 tokens/s generation

quant:

3-bit (gguf)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User quantized DeepSeek V4 Flash to custom 3-bit using llama.cpp PR #24162. Reports 5-6 tps, early stage with stability/performance tradeoffs. Praises model intelligence, efficiency with context window scaling, and low KV cache usage. Predicts dominance in 80-140GB model space.

reported speed:

49.4 tokens/s generation · 2050.0 tokens/s prompt processing

quant:

MXFP8 x MXFP4

kv:

fp8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Two ASUS GX10 (DGX Spark) units connected via RoCE, TP=2. Running DeepSeek-V4-Flash with MXFP8 x MXFP4 weights. Performance table at concurrency=1: 4K context prefill 2050 t/s decode 49.4 t/s; 16K 2150/43.0; 32K 2130/37.9; 128K 1920/42.5; 256K 1680/39.8. At C=4 128k aggregate ~40-42 t/s. Max context ~1M tokens in KV cache, typically runs 256k. Uses vLLM fork from local-inference-lab. Model praised for high context retrieval and reasoning, beats M2.7 and Stepfun 3.7 on private benchmark. Power ~280W at max load. Docker compose provided.

reported speed:

193.0 tokens/s generation

quant:

W4A16-FP8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Optimized DeepSeek V4 Flash on Hopper with Canada-Quant quants and patched MTP code in vLLM to achieve 193 tok/s. Also mentions hitting ~400 tok/s with 4 concurrent threads.

reported speed:

40.0 tokens/s generation

quant:

FP8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Dual DGX Sparks running DeepSeek V4 Flash at ~40 tk/s single, 350 tk/s aggregate with 32 concurrent requests at 256k context. Compared to RTX Pro 6000 (Q2 ~46 tk/s) and Mac M2 Ultra 192GB (Q2 ~29 tk/s).

reported speed:

12.7 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Running DeepSeek V4 Flash on M3 Max 96GB with ssd-streaming and iogpu.wired_limit_mb=86016. Generation speed ~11-13 t/s, TTFT 3-5s after warmup. Prefill of 36k tokens takes ~2.5 minutes. Uses antirez's ds4 engine and GGUF.

reported speed:

40.0 tokens/s generation · 1800.0 tokens/s prompt processing

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Two DGX Spark units running DeepSeek V4 Flash locally. Prefill 1800 t/s, generation 40 t/s. Praises scalability via ConnectX and power efficiency.

reported speed:

13.7 tokens/s generation · 159.0 tokens/s prompt processing

quant:

mixed Q8/Q4/Q2 (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Patched llama.cpp to support DeepSeek V4 Flash with full 1M context on RTX 5090. Prefill 159 t/s at 1M, decode 13.7 t/s. Peak VRAM ~31 GiB. Verified with needle-in-haystack test.

reported speed:

21.3 tokens/s generation · 1105.0 tokens/s prompt processing

quant:

Q2_K (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Optimization results for DeepSeek V4 Flash on RTX 5090 with llama.cpp fork. Reports TG t/s from 22.7 to 21.3 and PP t/s from 1105 to 927 across prompt lengths. Uses Q2_K quant, 1M context, and CPU offloading for MoE.

reported speed:

3.2 tokens/s generation

quant:

MXFP4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User complains about slow CPU inference with DeepSeek-V4-Flash in MXFP4, comparing to GLM 5.2 at 1.8 t/s. Mentions CPU-only mode due to weak GPU.

reported speed:

20.0 tokens/s generation · 533.0 tokens/s prompt processing

quant:

8bit (affine)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Optimized DeepSeek V4 Flash 8-bit affine MLX model on oMLX. Prefill improved from ~300-321 to ~533 tok/s, decode from ~7.31 to ~20-22 tok/s. Real runs at 79K-119K context show 19.2-20.7 tok/s. Seeking community review on accuracy and next optimization directions.

reported speed:

20.6 tokens/s generation · 534.9 tokens/s prompt processing

quant:

NVFP4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

REAP-pruned DeepSeek-V4-Flash on a single DGX Spark (Ascent GX10) served at 262k context with vLLM. Long-context consistency highlighted; prefill and generation throughput remain stable from 4K to 162K context. Benchmarks include pp and tg at various context lengths and concurrency levels.

reported speed:

8.0 tokens/s generation

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports 8 tokens/s on Mac Studio M3 Ultra 512GB with DeepSeek V4 Flash GGUF Q4_K_XL, expecting better performance. Also tried Q8.

kv:

fp8_e4m3

Deployment of DeepSeek V4 Flash with DSpark speculative decoding on HGX-H200 (4 GPUs, TP=4) via SGLang. Compares marlin and flashinfer MoE backends. DSpark is faster than EAGLE: 3.2x at bs=1, +46% throughput at bs=24. Reports TTFT and accept length benchmarks.

reported speed:

10.0 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports a fix for DeepSeek V4 Flash crash and stall in llama.cpp. The model is used for agentic coding with pi. The fix addresses a seq_rm issue causing stalls and crashes. User experience improved even at 10 t/s.

reported speed:

37.0 tokens/s generation · 443.0 tokens/s prompt processing

quant:

2bit

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Profiling llama.cpp led to 29x speedup on DeepSeek V4 Flash. Fixed CUDA attention kernel fallback, MoE tile padding, and used MTP head for decode. Prefill at 97k: 443 t/s; at 253k: 256 t/s. Decode: 37 t/s. Model is 284B in 2bit, 87GB, fits in 96GB VRAM.

reported speed:

128.6 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Running DeepSeek V4 Flash on a single RTX 6000 Pro using vLLM-Moet. Context length set to 130K. Requires ~150GB RAM for loading. Benchmarks show generation speeds around 100-130 t/s.

reported speed:

7.4 tokens/s generation · 15.2 tokens/s prompt processing

quant:

UD-Q2_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

98GB model running on 16GB VRAM + CPU offload. Speed improved from 2 to 7 t/s with recent llama.cpp updates.

reported speed:

17.5 tokens/s generation · 273.2 tokens/s prompt processing

quant:

IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8 (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Running on 2x RTX 3080 20GB with 64GB DDR5. Uses llama.cpp fork by fairydreaming. Achieves 17.53 tk/s generation and 273.21 tk/s prefill at 128k context. Model is 86.7 GB. Uses --n-cpu-moe 28 and split-mode layer.

reported speed:

6.6 tokens/s generation · 37.5 tokens/s prompt processing

quant:

Q4_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Budget build with dual Xeon E5-2660 v2 and RTX 3090. DeepSeek V4 Flash with Q4_K_XL quant, 8k context, pp 37.47 t/s, tg 6.62 t/s. Also tested Cydonia v4.3 (Mistral Small 3.2 finetune) with 308.92 t/s pp and 7.8 t/s tg.

reported speed:

17.0 tokens/s generation · 675.0 tokens/s prompt processing

quant:

Q8_K_XL (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Prefill ~650-700 t/s, decode ~17 t/s. Uses CPU offloading for MoE experts. Loading time 32 sec. User notes speed not as impressive as Qwen models but sees room for optimization.

reported speed:

11.4 tokens/s generation · 262.7 tokens/s prompt processing

quant:

IQ3_XXS-AS (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmark of DeepSeek V4 Flash GGUF in IQ3_XXS-AS and IQ2_S quants on RTX 3090 with 128GB DDR4. IQ3_XXS-AS preferred as sweet spot. Mainline llama.cpp b10064 matches fairydreaming fork performance.

quant:

IQ2_XXS/Q2_K (GGUF)

Compared DeepSeek-V4-Flash on M5 Max MacBook (128GB) with heavily quantized GGUF (IQ2_XXS/Q2_K, ~2.45 bits/weight) vs 2x DGX Spark with native FP8/FP4 checkpoint. Terminal-Bench 2.1: Mac 54% (47/87), Spark 52% (45/86). Mac-only wins: 11, Spark-only: 9. No significant difference in task-solving, but Spark pair offers higher speed (54-58 tok/s single-stream, 253 tok/s aggregate at concurrency 8) and larger context (262K server window).

reported speed:

9.1 tokens/s generation · 37.9 tokens/s prompt processing

quant:

Q8_K_XL (GGUF)

kv:

f16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User runs DeepSeek V4 Flash (unsloth Q8_K_XL GGUF) on a multi-GPU setup: two RTX 3090 24GB and one RTX 5090 32GB (total 80GB VRAM) with 128GB DDR4 RAM. Uses llama.cpp server with tensor splitting and offloading MoE layers to CPU. Reports 37.91 t/s prompt processing and 9.07 t/s generation. Mentions performance better than Qwen 3.5 v2 on coding tasks. Asks for speed improvement suggestions.

kv:

fp8

Single B300 GPU, vLLM 0.25.0, batch 256, ~300 output tokens per item. Config: tensor_parallel_size=1, block_size=256, max_num_seqs=256, enable_prefix_caching=true, moe_backend=flashinfer_trtllm, reasoning_parser=deepseek_v4, attention_config use_fp4_indexer_cache=true, compilation_config cudagraph_mode=FULL_AND_PIECEWISE. User reports deep_gemm_mega_moe requires expert parallel on single GPU; disabling DSpark speculative decoding doubled throughput.

reported speed:

105.0 tokens/s generation

quant:

IQ2XXS

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Two RTX 4090D 48GB GPUs (96GB total) with p2p patch. Re-implemented Blackwell-only kernels in Triton for sm89. Also benchmarked llama.cpp with IQ2XXS quant, 262k context, q8_0 KV cache. Mentions 2-3x speedup for parallel agentic workflows vs llama.cpp.

reported speed:

180.0 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Compared Claude Code, OpenCode, and Pi as harnesses for DeepSeek V4 Flash. Quality of diffs was the same across harnesses, but time and token usage varied widely. Claude Code took ~4x longer than the fastest. Model served via vLLM at ~180 tok/s.

reported speed:

32.0 tokens/s generation · 245.0 tokens/s prompt processing

quant:

ROCmFP2

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

DeepSeek V4 Flash with speculative draft on Strix Halo. Decode 32.0 tok/s with DSpark, 25.31 tok/s autoregressive. Prefill 245 tok/s sparse. Quantization ROCmFP2 mixed precision, ~2.88 bits per parameter. Draft model Q4RMFP4. Compared to LocalMaxxing entries: HipFire 18.99 tok/s, DwarfStar 15.6 tok/s.

reported speed:

35.0 tokens/s generation

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Post asks if anyone has run DeepSeek-V4-Flash-Strix-Halo-GGUF on a Strix Halo. Mentions it fits with ~64K context and allegedly 35 t/s. Tone is cautiously optimistic but not fully positive.

reported speed:

16.1 tokens/s generation

quant:

Q8_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

All experts on CPU, only 15.8GB of 40GB VRAM used.

reported speed:

30.0 tokens/s generation

quant:

Q4_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User asks about minimum VRAM GPU to run DeepSeek-V4-Flash-0731 Q4_K_XL at around 30 t/s. Mentions MoE with 13B active parameters and asks about GPUs up to 48GB VRAM. No specific hardware or engine mentioned.

reported speed:

31.5 tokens/s generation · 836.0 tokens/s prompt processing

quant:

Q8_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmark comparing TensorSharp vs llama.cpp on 4x A40 GPUs. TensorSharp cuda backend achieves 836 tok/s prefill at 16K context and 31.5 tok/s decode short. Model is DeepSeek-V4-Flash-0731-UD-Q8_K_XL from unsloth.

reported speed:

30.0 tokens/s generation

quant:

Q2-Q4 mixed imatrix (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User quantized DeepSeek V4 Flash for DS4 engine. Reports ~30 tok/s on M5 Max, double llama.cpp speed. Seeking feedback from CUDA/ROCm users.

reported speed:

11.0 tokens/s generation · 200.0 tokens/s prompt processing

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

4x RTX 5060 Ti 16GB, DDR4 3200 RAM at 4-channel, -ub/-b at 4096

reported speed:

27.2 tokens/s generation · 14860.0 tokens/s prompt processing

quant:

Q3_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports running DeepSeek V4 Flash 0731 with Q3_K_XL quant on dual GPU setup (RTX 6000 96GB + W7800 48GB) using llama.cpp server. Achieved 27.2 t/s generation speed. Mentions it's cheaper than K3 and GLM 5.2 but very good.

reported speed:

3.5 tokens/s generation · 3.0 tokens/s prompt processing

quant:

IQ2_M

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Dual RTX 3060 with RAM offloading. Prompt eval speed 3.0 tok/s, generation speed reported as 4.5 tok/s in summary but 3.5 tok/s in PowerShell; user trusts PowerShell. LM Studio failed to offload to second GPU, used Unsloth Studio. CPU: Ryzen 7500F, RAM 96GB 5600. Task: write a tetris game.

reported speed:

7.7 tokens/s generation · 116.0 tokens/s prompt processing

quant:

Q3_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

llama-bench results on 3x RTX 3090 with Q3_K_XL quantization. Model size 119.40 GiB, 284.33 B params. Prompt processing 116.04 t/s, generation 7.71 t/s.

reported speed:

12.5 tokens/s generation

quant:

UD-IQ3_S

kv:

fp16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Ran DeepSeek-V4-Flash-0731 UD-IQ3_S with llama.cpp loader in text-generation-webui. Used --n-cpu-moe 39 to offload MoE experts to system RAM. Loader estimates ~136 GB total, relying on 128 GB DDR5 at 5600 MHz.

reported speed:

44.0 tokens/s generation · 564.0 tokens/s prompt processing

quant:

UD-Q8_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmark on Bosgame M5 with RTX PRO 6000 Max-Q eGPU. Multiple quants tested: UD-Q8_K_XL (150.8 GiB, 44.0 t/s decode, 564 t/s prefill, draft acceptance 0.535), UD-Q4_K_XL (144.4 GiB, 48.4 t/s decode, 585 t/s prefill, draft acceptance 0.532), UD-Q2_K_XL (90.2 GiB, 59.5 t/s decode, 1513 t/s prefill, no drafter). Uses DSpark drafter ported from closed PR. Mixed GPU setup with CUDA0 and ROCm0.

reported speed:

13.9 tokens/s generation

quant:

IQ3_XXS (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Requantized only the 129 routed expert tensors of DeepSeek-V4-Flash-0731 to IQ3_XXS (down projections at IQ3_S), leaving attention/shared experts/router at Q8_0/BF16/F32. 111.37 GiB in four shards. KLD vs MXFP4 source: mean 0.2386 vs UD-IQ3_S 0.2936, top-1 84.65% vs 82.78%, delta PPL +0.536 vs +0.685; but 2.12 GiB larger and worse max KLD (12.53 vs 11.13). Speed on 5 mixed GPUs (2x 3090, 5060 Ti, 2x 4060 Ti, 96 GiB VRAM) with expert spill to CPU: 13.91/13.57/13.26 t/s at depths 0/4096/16384 vs 9.88/9.69/9.51 for MXFP4 source (~1.4x). antirez's flat Q2 is 80.76 GiB, ~98% resident, 30.27 t/s. Known DSV4 SWA/rollback stall issues in llama.cpp; patched build used.

reported speed:

15.5 tokens/s generation · 107.5 tokens/s prompt processing

quant:

UD-IQ2_M

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Ran DeepSeek V4 Flash 0731 locally on 3x AMD MI50 32GB GPUs (96GB total VRAM) with UD-IQ2_M quant. Model size 90.9GB fits fully in VRAM. Generation speed ~15-16 t/s stable, prompt processing ~105-110 t/s. Used llama-server (llama.cpp). Tested with a Rubik's cube HTML coding task. User impressed by ability to run locally.

reported speed:

17.2 tokens/s generation · 70.0 tokens/s prompt processing

quant:

Q8_K_XL

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Prompt processing high 70s t/s, down to mid 30s at 300k context. Full 1M context fits in 48GB VRAM but PP slow.

reported speed:

12.0 tokens/s generation · 141.4 tokens/s prompt processing

quant:

UD-IQ3_XXS

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Prompt processing high 140s t/s, dropping to mid 80s at 60k context. Generation ~11.9 t/s. Uses ROCm 7.2.4 backend. GPU combo: 1x 7900 XTX 24GB + 3x MI60 32GB (one MI60 failed).

reported speed:

4.8 tokens/s generation

quant:

2-bit dynamic

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Runs DeepSeek-V4-Flash 284B-A13B on 24GB M5 Pro with ~5.3GB memory usage, 4.8 tok/s. Also runs Gemma 4 26B-A4B and Qwen 3.6 35B-A3B. Engine is Mference, an open-source engine.

reported speed:

19.1 tokens/s generation · 1000.0 tokens/s prompt processing

quant:

2-bit (FP8)

kv:

fp8

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

DeepSeek-V4-Flash-0731 (155.43 GiB FP8, 48 shards) served on DGX Spark (GB10, aarch64, sm_121, 121.7 GiB unified memory, 128 GiB swapfile) with vLLM-Moet (vLLM v0.25.0 + patch) at 2-bit MoE experts. Prefill steady 1000 tps. Decode plain ~19 tok/s; with MTP head 26.6 tok/s (+48%). MTP vs plain aggregate: conc1 25.2 vs 19.1 (+31.5%), conc2 30.9 vs 26.4 (+17.3%), conc4 43.2 vs 45.6 (-5.3%). Per-request: conc1 25.2 vs 19.1, conc2 23.0 vs 17.7, conc4 14.5 vs 13.9. KV cache 4.56M tokens at 512K context, util 0.90. Boot ~10 min warm, 31-46 min cold. 2-bit planes 43 layers x 1.69 GiB ≈ 73 GiB. MTP head from ycui7/DeepSeek-V4-Flash-MTP. Detailed setup gotchas: build on ARM64, multi-arch Dockerfile tag, sm_120 cubins work on sm_121, swapfile in fstab, read-only mount issues, DELTA_GB=0.

reported speed:

18.0 tokens/s generation · 700.0 tokens/s prompt processing

quant:

Q8_K_XL (GGUF)

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

User reports 700pp/s and 18tg/s with llama.cpp using --cpu-moe on Threadripper 5965WX with RTX 5090. Mentions llama.cpp doesn't support native FP8 cache for DFV4 Flash, requiring FP16 cache. Speculator not working. Suggests building a rig with Epyc and 2x 5060 Ti.

reported speed:

43.0 tokens/s generation

quant:

MXFP4

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Started at 34 tok/s, ended at 43 tok/s. Cached tokens were default chat prompt, query was 13k.

reported speed:

1.0 tokens/s generation · 50.0 tokens/s prompt processing

quant:

4bit

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Running experiments with streamed experts trick on M5 Air 32GB. Prefill ~50 tps, decode ~1 tps. Model is ~300B total params (DeepSeek V4 Flash 0731).

reported speed:

30.2 tokens/s generation · 64.0 tokens/s prompt processing

quant:

Q8_K_XL

kv:

bf16

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Benchmark comparing Q8_K_XL vs IQ3_XXS on DeepSeek-V4-Flash-0731-UD. Q8_K_XL offloads ~40GB to system RAM. IQ3_XXS is 2.2x faster decode and 2.4x faster prefill, with equal correctness. Q8_K_XL has more empty-answer failures (8/125 vs 5/125). Reasoning budget 4096 eliminates empty answers. Disabling reasoning degrades trap handling. Q8_K_XL is slower but more reliable in some tasks.

reported speed:

15.0 tokens/s generation · 800.0 tokens/s prompt processing

kv:

fp8_ds_mla

Reported by the source; GPU count, offloading and concurrent requests can change this figure. Check the full setup before comparing.

Full 1M context on single RTX 5090 with CPU/RAM offloading. ~800 tps pp & 15+ tps decode. DSpark speculative decoding with dynamic acceptance rates. Two routed MoE layers GPU-resident.