llamaperf

Qwen3.8 27B

on NVIDIA RTX 5090 · VoidInfer · 3,072 ctx

Tone: mixedNVIDIA hardware
Oct 27, 2026
Throughput
193.0 t/s gen
Quant
EXL3 6 bpw (EXL3)
KV cache
INT4
VRAM reported
32 GB

Use cases

codingmathagenticlong-context

Summary

User benchmarks Qwen3.8-27B on an RTX 5090 in VoidInfer at 193 tok/s decode with a 3K prompt, against NInfer at 219 tok/s. VoidInfer runs EXL3 6 bpw with OSCAR INT4 KV cache; NInfer runs NVFP4 with FP8 KV. Both use a 208,896-token window, one request, 1,024 tokens out, prefill excluded. Decode falls to 127 tok/s at 200K context versus 153 tok/s for NInfer, a 34% drop against 30%. VoidInfer scores higher on two coding tests and on AIME 2026 at both 82K and 205K budgets, while NInfer is 10-17% faster on decode and about 2-2.5x faster on prefill.