llamaperf

Qwen3.8 27B Swift-1.5

on NVIDIA RTX 4080 · llama.cpp · 81,920 ctx

Tone: positiveNVIDIA hardware
Oct 12, 2026
Throughput
70.0 t/s gen
Quant
IQ3_XXS (GGUF)
KV cache
q8_0
System RAM
32 GB
VRAM reported
16 GB

Summary

User reports Qwen3.8 27B (Swift-1.5 fine-tune, IQ3_XXS GGUF) at 70 t/s on an RTX 4080 16GB with 81920 context, using MTP plus n-gram speculation. Setup is llama.cpp (llama-server CUDA build in WSL2/Debian) with q8_0 KV cache, flash attention on, all layers on GPU, and 14.9 GB of the 16 GB VRAM used. With n-gram only the user got 45 t/s at 204800 context. They note MTP was worth the tradeoff and could grow context further by dropping MTP or allowing spillover into system memory.