Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.8 Uncensored Q8 at 256k context on the AI PC (Strix Halo) at ~10 t/s.
Setup is a custom AI framework running across multiple devices. A Qwen3.6 35B MOE runs on an RTX 5090 at ~200 t/s, and a Gemma 4 model runs on a MacBook Air for security.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash on Strix Halo at 32.0 tok/s decode with DSpark speculative draft and 25.31 tok/s autoregressive.
Setup is ROCmFP2 mixed precision at roughly 2.88 bits per parameter, with a Q4RMFP4 draft model. Prefill runs at 245 tok/s sparse.
The user compares this to LocalMaxxing entries of 18.99 tok/s for HipFire and 15.6 tok/s for DwarfStar.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User asks whether anyone has run DeepSeek-V4-Flash-Strix-Halo-GGUF on a Strix Halo.
The user notes it fits with ~64K context and is reported at 35 t/s.
The user is cautiously optimistic about the result.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports DeepSeek V4 Flash 0731 at 26.76 t/s sustained decode and 236 t/s prefill on a Strix Halo (Ryzen AI MAX+ 395, Radeon 8060S).
Setup is llama.cpp Vulkan v0.6.1 with DSpark speculative decoding, a bf16 draft model, and a q8_0 KV cache.
The user compares this with the DGX Spark and notes that q8_0 KV beats f16, the draft must be bf16, Vulkan beats ROCm, ubatch tops out at 2048, and a v0.6 regression is fixed in v0.6.1.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports 24 t/s on a Ryzen AI Max+ 395 and 51 t/s on a Radeon AI PRO R9700.
The 51 t/s figure is attributed to the R9700, not the Strix Halo.
User asks about t/s for various quants and settings.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.8-27B Q8_0 on a Ryzen AI Max+ 395 in a ROG Flow Z13 with 128 GB unified memory, generating at 9-19 t/s and sustaining 16-19 t/s.
Setup is Lemonade Server with llama.cpp ROCm, a Q8 KV cache, native MTP speculative decoding, a 64 GB VRAM and 64 GB RAM split, and roughly 142k context.
MTP acceptance runs 97-99%. The model generated a flight simulator through an agent with file and bash tools.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a speculative decoding sweep with DSpark drafters, from a 20.48 t/s baseline to a peak of 28.5 t/s at n_max=3, a 1.39x gain.
Q2_K_S and Q8_0 drafters land within 1-3% of each other.
The user estimates 22-28 t/s for realistic mixed use, with prefill at 200-220 t/s at 64k context.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.8 Q4_K_M at 1M context on an AMD Strix Halo 128GB with an eGPU RTX 3080 Ti 12GB over Oculink, reaching about 45 t/s.
Setup splits layers: 2GB of weights and FastMTP on the 3080, the rest on the iGPU.
At 262K context, Q6 gives 10 t/s on the Strix alone, 24 t/s with MTP n=4, and about 28 t/s with FastMTP offloaded to the 3080. Q4_K_M at 262K context with 5GB of weights and FastMTP on the 3080 gives about 53 t/s. The user notes 60-70GB free on the AMD, enough to load an embedding and reranker, and compares to Qwen3.6-35B on 4x3090 at 450 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a dense 27B model on a Strix Halo APU, with a DFlash2 drafter giving +40% decode over MTP.
Setup is Nathan's Vulkan fork of llama.cpp, with Q5_K_XL decoding faster than Q4_K_XL due to higher draft acceptance.
Prefill is ~300 t/s at 3k context. FP4 builds are slower and worse quality.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codinglong-context
User reports 153.32 t/s at 32K context and 87.74 t/s at 200K context.
Setup uses a Q4 KV cache, speculative decoding with MTP and n-gram, and the vision encoder on the iGPU.
HumanEval scored 159/164 in 29.7 min.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agentic
User reports a hybrid attention model (Gated DeltaNet plus sparse QSA) with a 51B n-gram embedding table, which llama-bench reports as 176.94B all-in.
Setup is a 3-part GGUF, 87 GiB on disk, built from llama.cpp PR #27742 with a crash fix, on the Vulkan (RADV) backend.
The model uses about 91 GB resident at 131k context and loads in about 45s. The KV cache must stay F16, as quantized KV asserts.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8 27B dense at 10 t/s on Strix Halo 128GB.
User considers eGPU options with 32GB and 48GB of VRAM for faster dense model inference and larger models.
User mentions 0731 (DeepSeek V4 Flash) and dflash speculation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports a hybrid setup with Strix Halo 128GB and an R9700 32GB over x4 PCIe.
Dense layers, KV cache, and the MTP drafter run on the R9700, while routed experts run on the Strix, using a custom llama.cpp fork.
The user compares this to stock at 24 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
creative-writing
User reports average prefill and token generation throughput with an MTP draft model.
User finds quality superior to Qwen3.8-27B for architectural tasks.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
coding
User reports Qwen3.8-Flash-Next at Q8_0 generating for over 45 minutes because of excessive thinking even at low reasoning effort.
Setup is llama.cpp RPC on 2x Strix Halo 128GB.
User compares the run with Qwen3.8-27B and DeepSeek V4 Flash on the same hardware.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 84 t/s with 4 streams at 196K context on a model split across an iGPU (Vulkan) and an RTX 3090 Ti (CUDA).
The model has 512 experts per layer, 36 layers of gated DeltaNet, 12 layers of top-k sparse attention, a 26.8 GiB n-gram table, and a built-in MTP draft head, at 103.69 GiB.
Baseline on the iGPU alone is 22.2 tok/s. HumanEval+ scores 155/164, against 156/164 for Qwen3.8-27B on dual 3090 vLLM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports running the pi coding agent with llama.cpp on Strix Halo, decoding at 26+ t/s and prefilling at ~200 t/s in the fast band.
Setup uses Qwen3.8-27B with the UD-Q4_K_XL quant, a DFlash2 drafter, f16 KV cache and 256k context. A Flash-Next model with IQ4_XS quant and MTP is also mentioned.
The KV cache hit rate is ~94%.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B A3B prompt processing at 237.9 t/s at depth 0 on Strix Halo 128GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
tool-useagentic
User reports Liquid AI's LFM2.5-2.6B at 113 tok/s on a Ryzen AI Max+ 395, a CPU speed from the vendor's own benchmarks.
Setup is the official Q4_K_M GGUF of around 1.67 GB, a 2.69B parameter model with 128K context, using under 2.5 GB memory in the vendor's tests.
The vendor also reports 30 tok/s on a phone and 220 tok/s on an M5 Max, and the user notes independent results are needed. Benchmarks are ToolSandbox 77.83, IFBench 59.17, BFCLv4 56.88 and LiveCodeBench 59.41, and the model card does not recommend it for agentic coding.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6 35B at 50 t/s on Strix Halo.
Setup is the Q8_XL quant.
User emphasizes energy efficiency and versatility of Strix Halo compared to Nvidia cards.