Ornith1.5 35B (3B active)
on AMD Strix Halo 128GB · llama.cpp
Use cases
vision
Summary
User reports Ornith-1.5-35B-A3B at 72.6 t/s generation and 1504.9 t/s prefill on AMD Strix Halo 128GB.
Setup is llama.cpp-rocm with Q4_0_ROCMFP4 GGUF quantization, -ngl 99, benchmarked with llama-bench -p 256,2048 -n 256 -r 5.
Seven ROCmFPX quant tiers are provided; the Q4_0_ROCMFP4_FAST tier is the bare baseline. The model is a 35B MoE with ~3B active parameters and supports vision via bundled mmproj and MTP speculative decoding.