Llama 3.3 70B / Q4_K_XL / Vulkan native PP512 TG128 / runtime build unknown
AMD Strix Halo 128GB · llama.cpp / Vulkan KHR cooperative matrices / build unknown
- generation:
- 5.1 tokens/s
- prompt processing (prefill):
- 78.0 tokens/s
- quant:
- Q4_K_XL
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
First-party benchmark by Valérian de Thézan de Gaussan, published November 25, 2025: https://valerian.dtdg.fr/blog/2025/amd-strix-halo-ai-395-llm-benchmark/ . The author identifies the measured machine as a Framework Desktop with AMD AI Max 395+ and 128 GB LPDDR5, and the benchmark section explicitly says llama.cpp Vulkan on the Radeon 8060S iGPU, with no dedicated GPU. One participating integrated GPU is attributed to that bounded local test setup. There is no numeric GPU inventory or startup-device log; GPU core/CU variant details, CPU core count, BIOS and OS/driver versions are not provided. The supplied CPU-thread count is 6, not the physical CPU core count. Source model Llama 3.3 70B; quantization Q4_K_XL. Exact tested conversion and any fine-tune are not specified. Tested filename, model publisher, revision, checksum and exact artifact are unknown. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct is a verified base/model-family reference only and must not be treated as the measured quantized artifact. Source Size column is 39.73 GB; this is model storage size, not installed, loaded or peak RAM. Physical unified RAM is independently stated as 128 GB; VRAM carve-out and measured memory use are unknown. Native llama-bench protocol: llama-bench -p 512 -n 128 -t 6 -ngl 999. Source explicitly defines pp as prompt/context ingestion and tg as decode, and reports PP 78 tokens/s and TG 5.05 tokens/s for this row. PP512 and TG128 are separate synthetic phase tests; do not combine them into an end-to-end rate. The source command requests maximum GPU offload; actual layer placement is not logged. Vulkan KHR cooperative matrices are stated. Runtime build/commit, executable checksum and patches are unknown. No defaults from another llama.cpp revision are assumed. The publication provides rounded rates without raw phase durations, per-repeat samples, statistical uncertainty or repetition count. The 512-token prompt and 128-token generation lengths are the declared benchmark protocol, not raw token-counter observations. Timer implementation and exact warmup/cache handling cannot be independently pinned to an unknown build. Context depth/capacity, batch/ubatch, flash-attention mode, KV precision, power limits, hardware clocks, thermals, background loads and swap are unreported. Measurement date is unreported; publication date is not silently treated as run date. No measured TTFT or first-token clock is supplied. No quality/correctness test is reported. These are source-reported native throughput measurements, not fresh measurements or estimates. All eleven source model rows share this setup; each model/quant pair is kept separate. Structured PP/TG are rounded to the form’s 0.1 tokens/s step; exact source rates remain above.