Ornith1.5 reports

generation:

28.6 tokens/s

quant:

Q4_K_M (GGUF)

kv:

q8_0

flash attention:

on

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ornith 1.5 35B-A3B at 28.6 t/s on an RTX 3080 Laptop 8GB with 32GB system RAM. Setup is llama.cpp with a Q4_K_M GGUF, 65,536 context, q8_0 KV cache, flash attention on, all layers on GPU, and 36 MoE layers on CPU. The user finds the model overthinks, struggles to choose an approach, and recovers poorly from errors, though it can execute Unity tasks when explicitly told which tool to call.

generation:

100-110 tokens/s

prompt processing (prefill):

2000.0 tokens/s

quant:

Q4

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.5-35B at around 100-110 t/s generation and ~2000 t/s prefill on 2x RTX 5060 Ti 16GB (32GB total). Setup uses Q4 quantization; no engine or context length specified. Previously ran Qwen 27B Q4_K_M at around 50 t/s. User asks for fast model recommendations for bulk text extraction and whether a community leaderboard tracks local inference speeds.

generation:

24.0 tokens/s

quant:

Q4_K_M (GGUF)

kv:

q8_0

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ornith1.5 35B-A3B at 24 t/s on a laptop with 6GB VRAM. Setup is llama.cpp with the Q4_K_M GGUF, a q8_0 KV cache, and 100,000 context. Throughput drops to 7 t/s when the GPU reaches 87-88C and throttles, or when context usage reaches about 70%. The model was used to generate a playable browser game from a single prompt, with minor manual fixes.

generation:

64.0 tokens/s

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.

User reports Ornith1.5 35B A3B at 64 t/s after fixing the MTP head, up from 60 t/s, and 33% faster wall clock. The model is a splice of a trained MTP head onto an APEX requant. The user mentions running on a PC with a hackRF receiver and a 5 watt quansheng portable.

generation:

36.0 tokens/s

prompt processing (prefill):

950.0 tokens/s

quant:

Q6_K (GGUF)

kv:

Q8

Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.

User reports Ornith-1.5-9B at about 36 t/s generation and about 950 t/s prompt eval on an AMD RX 9060 XT 16GB with llama.cpp. Throughput drops to about 25 t/s generation and about 500 t/s prompt eval during long tasks. The run lasted about 3.5 hours on a coding agent task. The user contrasts this with Qwen 3.8 27B, which was frustratingly slow on the same hardware.