Qwen3.8 27B

on 4× NVIDIA RTX 5060 Ti 16GB · llama.cpp

Sep 7, 2026

Throughput

20.0 t/s gen · 1000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.

Quant

q8 (GGUF)

System RAM

256 GB

VRAM reported

16 GB

Summary

User reports Qwen3.8-27B q8 GGUF at 20 t/s tg and 1000 pp on a Threadripper rig with 4x RTX 5060 Ti 16GB, VRAM only. User also reports DeepSeek V4 Flash 0731 GGUF at 11 t/s tg and 200 pp with RAM offload. User asks about vLLM tensor parallelism, NVFP4, a CPU upgrade, and other models.

View source ↗