Qwen3.8 125B (6B active) Flash-Next
on 4× NVIDIA RTX 3090 · vLLM · 262,144 ctx
Sep 27, 2026
Use cases
codingagentictool-uselong-contextvision
Summary
User reports Qwen3.8-Flash-Next at 193.3 tok/s median on 4x RTX 3090 (96 GB VRAM, 31 GB host RAM).
Setup is vLLM with Intel AutoRound W4A16 experts (Marlin), BF16 attention, FP8 n-gram table, fp8_e5m2 KV cache, 262,144-token context, 6 seats, and albucino's INT4 MTP draft with 3 tokens and expert parallel.
Without the draft the same lane measured 55.8 tok/s; real prompts ran 109.5 tok/s prose, 108.4 chat, 145.8 code. Six parallel coding agents sustained 317 tok/s aggregate (570 peak burst, 53 tok/s per agent, TTFT 0.47 s); at gmu 0.95 six agents gave 252.6 tok/s aggregate and 48 tok/s per stream. A llama.cpp lane on the same box measured 96 to 102 tok/s on copy/edit tasks and 40 to 49 tok/s freeform, with ~311 tok/s prefill and two 262,144-token slots.