llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 4× NVIDIA RTX 3090 · vLLM · 262,144 ctx

Tone: positiveNVIDIA hardware
Sep 27, 2026
Throughput
193.3 t/s gen
Quant
W4A16 (AutoRound)
KV cache
fp8_e5m2
System RAM
31 GB
VRAM reported
96 GB

Use cases

codingagentictool-uselong-contextvision

Summary

User reports Qwen3.8-Flash-Next at 193.3 tok/s median on 4x RTX 3090 (96 GB VRAM, 31 GB host RAM). Setup is vLLM with Intel AutoRound W4A16 experts (Marlin), BF16 attention, FP8 n-gram table, fp8_e5m2 KV cache, 262,144-token context, 6 seats, and albucino's INT4 MTP draft with 3 tokens and expert parallel. Without the draft the same lane measured 55.8 tok/s; real prompts ran 109.5 tok/s prose, 108.4 chat, 145.8 code. Six parallel coding agents sustained 317 tok/s aggregate (570 peak burst, 53 tok/s per agent, TTFT 0.47 s); at gmu 0.95 six agents gave 252.6 tok/s aggregate and 48 tok/s per stream. A llama.cpp lane on the same box measured 96 to 102 tok/s on copy/edit tasks and 40 to 49 tok/s freeform, with ~311 tok/s prefill and two 262,144-token slots.