Ornith1.5 122B
on NVIDIA RTX 3090 · llama.cpp · 32,768 ctx
Summary
User reports Ornith-1.5-122B at 15.5 t/s decode and 548 t/s prefill on a single RTX 3090 at 32K context.
Setup is llama.cpp with Q4_K_XL GGUF and routed experts kept in system RAM via --n-cpu-moe 38, on a cloud pod with 32 vCPU EPYC 7763 and 125 GB RAM.
Decode ranged 12-22 t/s across runs; Q3 reached 26 t/s and 128K context fits with --n-cpu-moe 40. Q4 peaked at 55 GiB RAM and the pod cost $1.53.