llamaperf

Caphlon-2 27B

on NVIDIA T4 16GB · PrismML llama.cpp

NVIDIA hardware
Nov 22, 2026
Throughput
16.3 t/s gen · 277.9 tokens/s prompt processing (prefill)Prompt processing is how fast the input is read, not how long you wait for the first word. How to measure time to first token.
Quant
PTQ1_SB (GGUF)
VRAM reported
16 GB

Use cases

math

Summary

User releases Caphlon-2, a 27B model based on Qwen 3.8 27B and Ternary-Bonsai-2-27B, reporting 277.95 tok/s prompt processing (pp512) and 16.33 tok/s generation on a Tesla T4, and around 18 tok/s on an M2 MacBook Air. The 4.52 GiB (1.648 bpw) build uses PTQ1_SB ternary quantization with a trimmed 83,200-token vocabulary and trimmed FFN layers 44-61, and needs a drop-in patch for PrismML llama.cpp. Reported scores are 80.0% on GPQA-Diamond, 87.4% on Halu-270 with ITI steering (85.9% pure) and 80.0% on EN-40. A separate Caphlon-1 build (6.87 GiB, IQ2_S) runs on stock llama.cpp and Ollama.

Compare this run

Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.

Everything reported on the NVIDIA T4 16GB →Every Caphlon-2 report →Estimate Caphlon-2 on this card →