Qwen3 8B

User benchmarks Qwen3-8B on an RTX PRO 6000 Blackwell 96GB, reporting 86 tok/s single-stream decode in BF16 under vLLM. Setup is vLLM 0.27.1 with BF16 weights, greedy decoding, concurrency 32 for aggregate figures; SGLang 0.5.9 and llama.cpp CUDA builds were also tested. Aggregate throughput at concurrency 32 was 1,725 tok/s for vLLM, 1,327 for SGLang and 428 for llama.cpp; single-stream decode sat in an 83 to 96 tok/s band across all stacks. An FP8 checkpoint on vLLM reached 130 tok/s single-stream and 2,597 tok/s aggregate, with p50 latency down from 0.74 s to 0.49 s.