llamaperf

Qwen3.6 27B

on NVIDIA RTX 5090 · vLLM · 200,000 ctx

Tone: positiveNVIDIA hardware
May 7, 2026
Throughput
73.6 t/s gen · 2883.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
NVFP4 (safetensors)
KV cache
Q8
Flash Attention
on
VRAM reported
32 GB

Use cases

coding

Summary

User reports generation speed varying 59-111 tok/s with MTP enabled and 3 speculative tokens. Setup uses an fp8_e4m3 KV cache, with prefix caching tested. Mean MTP acceptance length is 2.28. A stability pass at 200k context completed 10/10 runs.