llamaperf

Unknown family 27B

on NVIDIA RTX 4090 · llama.cpp · 100,000 ctx

Tone: positiveNVIDIA hardware
Oct 1, 2026
Throughput
80.0 t/s gen
Quant
Q4 (GGUF)

Summary

User reports a llama.cpp config tuner that measured a 27B Q4 model on an RTX 4090, improving average generation from 60 to 80 tok/s and peak from 80 to 110 tok/s. Setup is llama.cpp with Q4 weights and a 100K context length; the tuner sweeps speculative decoding, KV cache quant, GPU offload layers, and batch/ubatch, running 3 passes and taking the median. At 100K context the tuned config reached 60 tok/s versus 40 hand-tuned. The user says speculative decoding was the biggest single win and asks whether it helps as much on a 4090 as on a 5090.