llamaperf
Sep 18, 2026
Throughput
100-110 t/s gen · 2000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4
VRAM reported
32 GB

Summary

User reports Ornith-1.5-35B at around 100-110 t/s generation and ~2000 t/s prefill on 2x RTX 5060 Ti 16GB (32GB total). Setup uses Q4 quantization; no engine or context length specified. Previously ran Qwen 27B Q4_K_M at around 50 t/s. User asks for fast model recommendations for bulk text extraction and whether a community leaderboard tracks local inference speeds.