Qwen3.8 27B

on 2× NVIDIA RTX 3060 12GB · llama.cpp · 77,000 ctx

Sep 7, 2026

Throughput

26.9 t/s gen · 299.3 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

Q4_K_XL (GGUF)

KV cache

System RAM

32 GB

VRAM reported

22 GB

Use cases

coding

Summary

User reports a llama.cpp RPC cluster with tensor-split 20,23. Setup uses MTP speculative decoding with 84.37% draft acceptance and a 96.14% prefix cache hit ratio. Max context was stress-tested at 72,712/77,000 tokens.

View source ↗