Qwen3.8 27B
on NVIDIA RTX 5090 · NInfer · 240,000 ctx
Summary
User reports Qwen3.8 27B averaging in the 170s t/s on an RTX 5090, peaking at 220 t/s.
Setup is ninfer-serve with the nvfp4 weights, MTP speculative decoding with 3 draft tokens, lm-head-draft, an fp8 KV cache and 240,000 context.
User calls it double or more the throughput of llama.cpp.