llamaperf

Qwen3.8 27B

on NVIDIA RTX 4090 · llama.cpp · 194,048 ctx

Tone: positiveNVIDIA hardware
Sep 7, 2026
Throughput
40.0 t/s gen · 1650.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_K_M (GGUF)
KV cache
Q8
System RAM
48 GB
VRAM reported
24 GB

Summary

User reports 194,048 context tokens at ~1650 prompt t/s and ~40 t/s generation without MTP, and 131,584 context tokens at ~1650 prompt t/s and ~65 t/s generation with MTP, up to 80 t/s for code, on an RTX 4090 FE with 48 GB DDR4 in WSL2. Setup uses a q8_0 KV cache and q4_0 draft KV cache, with --cache-ram 32768, --ctx-checkpoints 32, --no-mmap and --mlock.