llamaperf

Qwen3.8 27B

on NVIDIA RTX 5090 · SparkInfer · 262,144 ctx

Tone: positiveNVIDIA hardware
Oct 6, 2026
Throughput
92.9 t/s gen
Quant
NVFP4 (NVFP4)
KV cache
int8
VRAM reported
32 GB

Use cases

codingtool-useagenticlong-contextvision

Summary

User reports Qwen3.8-27B at 92.9 tok/s decode on a single RTX 5090 with SparkInfer and no speculation. Setup is SparkInfer v0.5.5 with an NVFP4 checkpoint (17.92 GB, two shards) and int8 KV cache at the full 262,144-token context, which costs 27.9 GB. The same weights reach 85.8 tok/s on SGLang with FP8 KV, 161.7 tok/s on SGLang with the DSpark v2 drafter, and 264.8 tok/s overall (up to 420.2 on code) in SparkInfer's bench harness with DSpark. Under concurrency SparkInfer peaks at 344.3 tok/s aggregate at 8 requests, while vLLM reaches 546.6 tok/s at 16. A configured 32,022-token prefix cache cuts a repeated request from 5.75 s to 0.30 s.