Ornith1.5 35B (3B active)
NVIDIA RTX 3080 Laptop 8GB · llama.cpp · 65,536 ctx
- generation:
- 28.6 tokens/s
- quant:
- Q4_K_M (GGUF)
- kv:
- q8_0
- flash attention:
- on
agentictool-use
User reports Ornith 1.5 35B-A3B at 28.6 t/s on an RTX 3080 Laptop 8GB with 32GB system RAM. Setup is llama.cpp with a Q4_K_M GGUF, 65,536 context, q8_0 KV cache, flash attention on, all layers on GPU, and 36 MoE layers on CPU. The user finds the model overthinks, struggles to choose an approach, and recovers poorly from errors, though it can execute Unity tasks when explicitly told which tool to call.