Ornith1.5 35B (3B active)

Source and attribution: u/LemesoftNostalgic, "Running some unsloth models on a dual-RTX 3060 workstation — exact setup + tok/s numbers", https://www.reddit.com/user/LemesoftNostalgic/comments/1wxc5tw/running_some_unsloth_models_on_a_dualrtx_3060/ . Primary source re-opened through the web reader on 2026-10-13; the returned primary-page snapshot was labelled crawled three days earlier. No direct live-HTTP timestamp is claimed. The page has only a relative post age; exact publication and measurement dates are unknown. This is an author-reported native benchmark, not an independently repeated run. Model: Ornith 1.5 35B-A3B, GGUF, table quant IQ4_XS; source model-file size 16.94 GiB, source quant density 4.25 bpw. The author explicitly says this is their own Ornith quant, similar to Unsloth UD-IQ4_XS. Do not label an Unsloth or ornith-ai GGUF as the measured artifact. The local serving filename is Ornith-1.5-35B-IQ4_XS.gguf; no public download, revision or checksum is established. Upstream/base-model reference only: https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B . Tested GGUF revision and checksum are unknown. Hardware: two NVIDIA RTX 3060 GPUs totaling 24 GB nominal VRAM (2 x 12 GB mapping), each reported at PCIe Gen3 x16; HP Z440 workstation, Intel Xeon E5-2690 v3 (12 cores / 24 threads), 96 GB ECC DDR4 in four channels, 256 GB NVMe plus 1 TB SATA, 700 W PSU. Xubuntu 24.04 LTS, NVIDIA driver 580.173.02 and CUDA 13.0 are source-reported system details. Neither GPU power limit nor actual benchmark resident RAM/VRAM is reported. Native benchmark results: PP512 1870 t/s and TG128 97.5 t/s. The author explicitly defines pp512 as llama-bench prompt processing and tg128 as llama-bench token generation. These are separate tests with 512 newly processed prompt tokens and 128 generated tokens respectively, not a single 512-input/128-output application request. Repetitions, statistic, variation, raw timings, exact benchmark invocation, backend, fork/build/commit, thread counts, batch/microbatch, KV types, flash-attention state, existing cache depth and configured maximum context are unreported. The separate Serving section mentions beellama.cpp, llama-swap, the pi coding agent, 180224-token server context and a 131072 pi maxTokens setting. Its server commands, quantized-cache options, tensor splits and speculative/MTP flags must not be assigned to the PP512/TG128 measurements. In particular, 180224 is not a verified benchmark context length. The separate fib column is llama-cli Fibonacci with MTP decode; its 115.2 t/s is excluded from the native PP/TG headline. Memory and comparability: the GiB value is a model-file size, not measured resident usage. The model-file size being smaller than combined VRAM does not establish full GPU residency or the benchmark split mode; both placement and resident usage remain unreported.