Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 76.4 t/s generation and 319.9 t/s prompt processing on an M4 Max 64GB. Setup is llama.cpp with UD-Q4_K_XL, one Mac Studio with 40-core GPU and 64 GB unified memory. The headline Quick Chat run used 35 prompt tokens and 96 output tokens, with three measured samples per scenario; these are short inputs, not filled 4K or long-context benchmarks.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 79.9 t/s generation and 91 t/s prompt processing on an M4 Max with 64GB unified memory.
Setup is ZINC on Metal with the UD-Q4_K_XL GGUF, speculative decoding disabled, context auto-sized from device memory.
The headline Quick Chat run used 36 prompt tokens and 96 output tokens; the three rate samples span 78.71 to 79.95 t/s for generation and 90.1 to 92.7 t/s for prefill.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 124.3 t/s generation and 3342.7 t/s prompt processing on a single Apple M5 Max with 48 GB unified memory.
Setup is llama.cpp build 11146 with Metal, GGUF Q4_K_M weights, n_gpu_layers=99 and flash attention enabled.
The figures are separate llama-bench microtests, tg128 and pp512, not one 512-input/128-output request or server concurrency throughput; context length is unspecified.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 151.1 t/s generation and 3333.4 t/s prompt processing on an M5 Max 48GB.
Setup is MLX with 4-bit weights, batch size 1, unquantized KV cache and no speculative decoding, on one Mac Studio with 18 CPU cores and 40 GPU cores.
The figures are arithmetic means of five trials from mlx-lm 0.31.3; the prompt rate is a native engine phase rate with first-token work included, not client TTFT.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 130.4 t/s generation and 7154.6 t/s prompt processing on an M5 Ultra 256GB.
Setup is MLX with a 4-bit quant, batch 1 on one device, at a 32768-token input and 128-token output.
The figures are separate five-trial medians, not pooled across lengths; the 32768-input run peaked at 22.079GB of allocator high-water memory.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 89.3 t/s generation and 2258.1 t/s prompt processing on an M3 Ultra 512GB.
Setup is MLX with a 4-bit quant, batch 1 on one device, at 32768 input tokens and 128 output tokens.
The figures are separate five-trial medians, not pooled across lengths; the 32768-token run peaked at 23.999GB of allocator memory.
2× NVIDIA RTX PRO 5000 Blackwell (one 72GB card and one 48GB card) · llama.cpp
generation:
208.7 tokens/s
prompt processing (prefill):
7492.8 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 208.7 t/s generation and 7492.8 t/s prompt processing on one RTX PRO 5000 Blackwell 72 GB plus one RTX PRO 5000 Blackwell 48 GB.
Setup is llama.cpp CUDA b10941 with Q8_0, f16 K/V cache, flash attention on, batch 2048, microbatch 512, and tensor 1/1 split across the two cards.
The 208.7 t/s is TG128 and the 7492.8 t/s is PP512, separate tests, each the mean of five repetitions; the author reports TG reproducibility around 1% and PP drift up to about 10% between sessions.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B Q8_0 at 198.0 t/s generation and 5772.2 t/s prompt processing on one NVIDIA RTX PRO 5000 Blackwell 72 GB.
Setup is llama.cpp CUDA b10941 with Q8_0, f16 K and V cache, flash attention on, batch 2048, microbatch 512, and the whole model on GPU0.
The 198.0 t/s is TG128, the mean of five repetitions; the 5772.2 t/s is PP512, a separate test, not a combined request. The author reports generation reproducibility around 1% and prompt-processing drift up to about 10% across sessions.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.6-35B-A3B at 35 t/s on an RX 7800 XT 16GB, with a Ryzen 5 7600X and 32GB DDR5.
Setup is llama.cpp with UD-Q5_K_M (about 25GB), which spills past the 16GB card.
User asks whether to swap to a used 7900 XTX 24GB or add a used 7600 XT 16GB, and requests real t/s numbers for either setup under Vulkan or ROCm.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks a new strix-llama.cpp branch (rdna35-perf) on a Strix Halo 128GB box, reporting prompt processing 21-54% faster than Vulkan and 8-174% faster than stock ROCm, with generation tying Vulkan on the dense 27B and 2-3% behind on MoE and 2B at short context.
Setup is llama-bench with Q8_0 weights, q8_0 KV cache, flash attention on, ROCm/HIP backend, ub 4096 for the MoE. The headline Qwen3.6 35B-A3B figures are pp4096 4045 t/s and tg64 56.3 t/s at depth 0.
Quality checks show KL divergence closer to stock ROCm than Vulkan and unchanged perplexity. The branch is ROCm/HIP only and was tested on three models: Qwen3.6 35B-A3B, Qwen3.8 27B, and Qwen3.5 2B.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 49.1 t/s generation and 1028.1 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is llama.cpp with UD-Q4_K_XL, f16 key/value cache and FlashAttention on, measured at a depth of 16,384 already-prefilled KV tokens.
These are separate llama-bench pp512 and tg128 tests, not a fresh 16,384-token ingest; the same ladder gives 52.73 t/s generation at depth 0 and 50.95 t/s at depth 8,192.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 39.9 t/s generation and 1068 t/s prompt processing on an AMD Strix Halo 128GB box with a Radeon 8060S.
Setup is llama.cpp on the rdna35-perf ROCm/HIP branch with Q8_0 weights, q8_0 KV cache, Flash Attention on and microbatch 4096, measured with llama-bench.
Both figures are separate benchmark cells at 128k tokens of cache depth, not one end-to-end request; the prompt test processes 4096 additional tokens at that depth, and configured context is unspecified.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 61.7 t/s generation and 276.8 t/s prompt processing on a Mac mini with Apple M2 Pro and 32 GB unified memory.
Setup is mlx-lm 0.31.3 with MLX 0.31.2, MLX 4-bit group-64 weights of about 19.0 GiB, temperature 0 and --max-tokens 128.
The figures are medians of three runs on a 490-token prompt with 128 tokens generated; peak memory was 20.261 GB per run.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 45.6 t/s generation and 1339.4 t/s prefill on an AMD Strix Halo 128GB system.
Setup is llama.cpp with Q8_K_XL and Flash Attention, using a patched ROCm 7.2.3 container with -ngl 99, -fa 1, -mmp 0, -dio 1, -b 2048, -ub 2048.
The headline combines separate pp512 and tg128 phase tests, not one paired request; the baseline control reports 1083.55 t/s prefill and 45.82 t/s generation.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 90 tok/s on an M4 Pro Mac mini with 48GB unified memory.
Setup is oMLX 0.7.0 with a custom oQ4 quantised model using MTP headers.
User notes performance had recently fallen as low as 10-11 tok/s before this improvement.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 24.53 t/s decode and 256.80 t/s prefill on an 8K prompt with an optimized Strata build on a GTX 1050 Ti Mobile 4GB.
Setup is Strata with Unsloth UD-IQ3_XXS GGUF and MTP speculative decoding, on an Intel Core i7-8750H with 48 GB RAM, Windows 11, CUDA 12.9.
Compared with llama.cpp b11524 (7.88 t/s decode, 113.47 t/s prefill at 8K) and the original Strata compatibility build (15.00 t/s decode, 141.75 t/s prefill at 8K). At 16K the optimized build reaches 21.40 t/s decode and 210.98 t/s prefill. Figures are averages of four runs per prompt size with a fixed 1,000-token output target.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-use
User reports Qwen3.6-35B-A3B at 3-5 t/s on a Radeon RX 6600 8GB with 64 GB DDR4 2800 and PCIe 3.0. The model was run with Q5_K_M quantization and required CPU offloading, which the user notes is as slow as running entirely on CPU due to slow PCIe and RAM.
Setup is llama.cpp with GGUF Q5_K_M, offloading to CPU. The user also tested qwen2.5-coder-7b-instruct-q5_k_m (good speed but limited context up to 24k) and DeepSeek-R1-Distill-Qwen-7B-Q4_K_M for general chat.
User asks whether 8GB VRAM is sufficient for complex tasks and what others use their local AI for. Sentiment is mixed: some models work but are too slow or limited for real-world use.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 73-75 tok/s generation and 1,390 tok/s prompt processing on a laptop with an RTX 4070 8 GB, i7-14700HX and 32 GB RAM.
Setup is PolyStrata with UD-Q3_K_M GGUF, a 30,000-token prompt, CPU offload of experts and speculative decoding.
The same model on an RTX 5090 32 GB writes 332-334 tok/s and reads 5,520-5,580. Gemma 4 26B-A4B, Ornith 1.5 and GLM 5.3 Flash were also measured on both machines; GLM 5.3 Flash does not fit in 32 GB of RAM.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
agenticcodingtool-use
User asks how to combine a 128GB Evo-X2, a 48GB MacBook Pro M5 Pro and an RTX 4080 Super desktop for local LLM work.
The Evo-X2 runs a Qwen 35B MoE at about 500-900 tok/s prompt processing and roughly 50 tok/s decode, used for business products rather than as a daily driver.
The user also describes a memory layer and an orchestrator setup with leads and an oracle, and asks whether they are reinventing the wheel.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 140 t/s decode around 8K context on an RTX 4070 Super 12GB.
Setup is Polystrata with a HauhauCS uncensored Aggressive TTQ Q4_K_M GGUF and MTP enabled, on 32GB DDR5-6000 RAM under Windows 11.
Prompt processing was 2,800-3,300 tok/s, and decode fell to 100 tok/s around 32K context. The user compares against llama.cpp at 2600 tok/s prefill and 77 tok/s decode, a roughly 1.3x prefill and 82% faster decode gain.