Caphlon-2 27B
NVIDIA T4 16GB · PrismML llama.cpp
- generation:
- 16.3 tokens/s
- prompt processing (prefill):
- 277.9 tokens/s
- quant:
- PTQ1_SB (GGUF)
User releases Caphlon-2, a 27B model based on Qwen 3.8 27B and Ternary-Bonsai-2-27B, reporting 277.95 tok/s prompt processing (pp512) and 16.33 tok/s generation on a Tesla T4, and around 18 tok/s on an M2 MacBook Air. The 4.52 GiB (1.648 bpw) build uses PTQ1_SB ternary quantization with a trimmed 83,200-token vocabulary and trimmed FFN layers 44-61, and needs a drop-in patch for PrismML llama.cpp. Reported scores are 80.0% on GPQA-Diamond, 87.4% on Halu-270 with ITI steering (85.9% pure) and 80.0% on EN-40. A separate Caphlon-1 build (6.87 GiB, IQ2_S) runs on stock llama.cpp and Ollama.