Qwen3.8 27B
NVIDIA RTX 4080 · NInfer · 8,192 ctx
- generation:
- 166.7 tokens/s
- prompt processing (prefill):
- 2719.9 tokens/s
- quant:
- GSQ (safetensors)
- kv:
- rk4v4-e8
coding
User reports Qwen3.8 27B at 166.7 t/s generation and 2719.9 t/s prefill on an RTX 4080 16GB. Setup is NInfer with GSQ safetensors and rk4v4-e8 KV cache, single card, DFlash2 speculative decoding at 8K context. User also reports MTP3 decode at 151.2 t/s and DFlash2 K=7 decode at 262.3 t/s at 32K, with prefill up to 2720 t/s and 100k context support.