Qwen3.8 27B

on 2× NVIDIA RTX 5060 Ti 16GB · LM Bionic · 200,000 ctx

Sep 7, 2026

Throughput

46.0 t/s gen · 700.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K_XL (GGUF)

KV cache

System RAM

32 GB

VRAM reported

16 GB

Summary

User reports 46 t/s generation and 700 t/s prefill with tensor parallelism enabled, against 28 t/s generation and 1000+ t/s prefill without it. Setup uses MTP with max draft tokens 6 and probability 0.88. Draft acceptance is 0.93757 with a mean length of 5.10.

View source ↗