Caphlon-2 27B
on NVIDIA T4 16GB · PrismML llama.cpp
Use cases
math
Summary
User releases Caphlon-2, a 27B model based on Qwen 3.8 27B and Ternary-Bonsai-2-27B, reporting 277.95 tok/s prompt processing (pp512) and 16.33 tok/s generation on a Tesla T4, and around 18 tok/s on an M2 MacBook Air.
The 4.52 GiB (1.648 bpw) build uses PTQ1_SB ternary quantization with a trimmed 83,200-token vocabulary and trimmed FFN layers 44-61, and needs a drop-in patch for PrismML llama.cpp.
Reported scores are 80.0% on GPQA-Diamond, 87.4% on Halu-270 with ITI steering (85.9% pure) and 80.0% on EN-40. A separate Caphlon-1 build (6.87 GiB, IQ2_S) runs on stock llama.cpp and Ollama.
Compare this run
Medians of plain runs: one device, one request, no speculative decoding, the whole model in the device's memory. The report above is one setup, so read it beside them and not as one of them.
Everything reported on the NVIDIA T4 16GB →Every Caphlon-2 report →Estimate Caphlon-2 on this card →