Qwen3.8 27B

on NVIDIA RTX Pro 6000 Blackwell · NInfer · 262,000 ctx

Sep 7, 2026

Throughput

104.8 t/s gen · 12400.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

groupwise-int

VRAM reported

96 GB

Use cases

agentictool-uselong-context

Summary

User reports Qwen3.8-27B on an RTX PRO 6000 Blackwell at 104.83 tok/s weighted decode, running a long-horizon agentic task at 262K context. Setup is NInfer with a groupwise-int quant (mixed Q4/Q5/Q6). The run covered 966 model calls, 131.2M input tokens and 853.3K output tokens. Prompt processing was about 12.4K tok/s, measured client-side and including queue time. The user plans to try an NVFP4 profile and estimates API cost savings of $650+.

View source ↗