Qwen3.8 27B
on NVIDIA RTX 5090 · Token Rush · 200,000 ctx
Summary
User reports Token Rush decoding Qwen3.8-27B at 70 tok/s without speculation at 200k tokens of context on one RTX 5090.
Setup is Token Rush with int4 GPTQ weights at 4.25 bits per weight and a 128-group scale, FP8 KV cache, speculation off for this figure, one stream, the whole model in VRAM.
With DFlash2 speculation the same engine decodes 229 tok/s on essay, 358 on code and 379 on math at short context, and 200-220 tok/s on prose at 200k. The full 256k window fits in 26 GB of VRAM.