Qwen3.8 125B (6B active) Flash-Next

User reports Qwen3.8-Flash-Next (125B MoE + 51B n-gram table) at 32 t/s decode and 463 t/s prefill on 2xRTX 3090 + 96GB DDR5. Setup is llama.cpp with experts offloaded to RAM and the n-gram table offloaded to NVMe, UD-Q4_K_XL, a 640K serving pool (4x160K lanes, q8 KV), and the 51B n-gram/PLE table mmap'd on NVMe. The MoE expert cache (RFC #24528 branch) at explicit --moe-cache 5000 gives +30% decode, while auto grants nothing. Concurrency erases the gain, so the user fans out shallow and serializes deep. Prefix caching reprocesses ~24 tokens on warm chat turns, with checkpoints anchored at turn boundaries and DeltaNet state included. The user found and fixed a gridDim.y overflow crash (deep multi-stream batches, QSA indexer norm) with a 2-line reorder, the same shape as the pending upstream fix in PR #27941. Serving holds 32 t/s while pulling 90GB via cgroup write-caging (systemd-run --scope -p MemoryMax=6G). In vLLM, the user grafted the unmerged mmap-PLE PR (#54129) onto the qwen38-flash-next image, serving the table from NVMe under W4A16, TP=2, in 53GB where the official recipe wants ≥110GB free RAM. Real-table decode is 15.3 t/s vs 16.0 with the table stubbed, so the NVMe n-gram costs ~4%. Selective experts-only UVA offload is mandatory: naive --cpu-offload-gb recopies the whole offload every step (2.5 t/s), while --cpu-offload-params routed_experts.*_weight_packed gives 16 t/s. MTP speculation hurts when the PCIe bus is the bottleneck (13.9 vs 16.0). vLLM is currently being optimised and tested for concurrency, and single stream is behind llama.cpp right now.