Ornith1.5 35B (3B active)
on NVIDIA RTX 4070 Ti Super · llama.cpp · 65,536 ctx
Use cases
coding
Summary
User reports Ornith-1.5-35B-A3B Q4 at 86–91 t/s generation and about 1,030 t/s prompt processing on an RTX 4070 Ti SUPER 16 GB, at a roughly 3–6k-token prompt.
Setup is llama.cpp b11370 (CUDA) in router mode with Q4_K_M weights, q8_0 KV cache, 65536 context, flash-attn on, and n-cpu-moe 15, using about 15.1–15.6 GB of the 16 GB card.
The figures are given as ranges, so no single throughput is recorded. The run is compared against a previous RTX 3060 Ti 8 GB setup; the host is an i7-14700K with 32 GB RAM on Windows 11.