llamaperf

Ornith1.5 35B (3B active)

on NVIDIA RTX 4070 Ti Super · llama.cpp · 65,536 ctx

NVIDIA hardware
Oct 20, 2026
Throughput
86-91 t/s gen
Quant
Q4_K_M (GGUF)
KV cache
q8_0
Flash Attention
on
System RAM
32 GB
VRAM reported
16 GB

Use cases

coding

Summary

User reports Ornith-1.5-35B-A3B Q4 at 86–91 t/s generation and about 1,030 t/s prompt processing on an RTX 4070 Ti SUPER 16 GB, at a roughly 3–6k-token prompt. Setup is llama.cpp b11370 (CUDA) in router mode with Q4_K_M weights, q8_0 KV cache, 65536 context, flash-attn on, and n-cpu-moe 15, using about 15.1–15.6 GB of the 16 GB card. The figures are given as ranges, so no single throughput is recorded. The run is compared against a previous RTX 3060 Ti 8 GB setup; the host is an i7-14700K with 32 GB RAM on Windows 11.