Qwen3.8 27B
on NVIDIA RTX 5090 · SparkInfer · 262,144 ctx
Use cases
codingtool-useagenticlong-contextvision
Summary
User reports Qwen3.8-27B at 92.9 tok/s decode on a single RTX 5090 with SparkInfer and no speculation.
Setup is SparkInfer v0.5.5 with an NVFP4 checkpoint (17.92 GB, two shards) and int8 KV cache at the full 262,144-token context, which costs 27.9 GB.
The same weights reach 85.8 tok/s on SGLang with FP8 KV, 161.7 tok/s on SGLang with the DSpark v2 drafter, and 264.8 tok/s overall (up to 420.2 on code) in SparkInfer's bench harness with DSpark. Under concurrency SparkInfer peaks at 344.3 tok/s aggregate at 8 requests, while vLLM reaches 546.6 tok/s at 16. A configured 32,022-token prefix cache cuts a repeated request from 5.75 s to 0.30 s.