llamaperf
Oct 4, 2026
Throughput
72.6 t/s gen · 1504.9 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q4_0_ROCMFP4 (GGUF)
System RAM
128 GB

Use cases

vision

Summary

User reports Ornith-1.5-35B-A3B at 72.6 t/s generation and 1504.9 t/s prefill on AMD Strix Halo 128GB. Setup is llama.cpp-rocm with Q4_0_ROCMFP4 GGUF quantization, -ngl 99, benchmarked with llama-bench -p 256,2048 -n 256 -r 5. Seven ROCmFPX quant tiers are provided; the Q4_0_ROCMFP4_FAST tier is the bare baseline. The model is a 35B MoE with ~3B active parameters and supports vision via bundled mmproj and MTP speculative decoding.