Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 82 t/s decode on a 4x RTX 3090 rig with MTP disabled.
Setup is a Q6 quant with MTP speculative decoding off; the same rig reached about 70 t/s with MTP enabled.
The user attributes the gain to MTP's draft-and-verify step needing extra sync passes across the four layer-split GPUs, where PCIe hops cost more than the accepted draft tokens save.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
agenticcoding
User reports Qwen3.6-35B-A3B at about 90 tokens/sec on an RTX 3070 Laptop 8GB with 32GB DDR4 and an i7-11800H CPU.
Setup is PolyStrata with a Q4_K_M GGUF quant; the model is larger than the card's 8GB, so weights ran partly from system RAM.
The user describes the low-cost configuration as working quite well for defining tasks, with results that look okay.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports 27.01 t/s generation and 517.25 t/s prompt processing.
Source evidence (verbatim):
Source: https://note.com/ripe_hosta6920/n/nb81b72d54ad7?hl=en
# First-hand RTX 3070 8GB Qwen3.6-35B-A3B CPU-MoE tuning benchmark
Author try tested May 10, 2026; published May 13. Hardware: NVIDIA RTX 3070 8GB VRAM, Ryzen 7 3700X (8 CPU cores), 32GB DDR4 system RAM, native Windows. Runtime llama.cpp b9119-ef93e98d0, CUDA 13.1 build; installed CUDA 13.2. Model Qwen3.6-35B-A3B, Unsloth UD-IQ4_XS GGUF, approximately 17.7GB disk size, not resident RAM. Verified matching publisher repository: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF ; tested revision unreported.
Selected measured llama-bench row: generation 27.01 ±1.34 tokens/s (tg128); prefill 517.25 ±34.22 tokens/s (pp512). Separate synthetic PP512 and TG128 tests, not an end-to-end 512-input/128-output request. Settings: ncmoe 27, 9 threads, flash attention enabled, K/V cache q4_0; CPU MoE offloading is essential. Source shows server recipe ngl 999, no-mmap, context capacity 8192; 8192 is not measured prompt length. Repetition count, cache reuse, benchmark command details, speculative decoding and concurrency unreported. Resident RAM and exact VRAM usage unreported (author says nearly full).
Other sweep rows and real-prompt Ollama comparison are distinct tests, not combined here. RTX3060 figures are quoted from another author and excluded. No Reddit material was accessed.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
codingagentic
User reports Qwen3.6-35B-A3B at 38.0 tok/s on a Mac mini M4 16GB, running a four-agent CrewAI coding crew (analyst, architect, developer, tester) via Coachwhip v0.5.3.
Setup is Coachwhip with UD-IQ2_M quant (11.5 GB) and all experts in GPU memory; temperature 0.7, top-p 0.8. The developer agent wrote 1,069 tokens at 38.0 tok/s and read 810 tokens at 274 tok/s; the tester wrote 1,811 tokens at 36.4 tok/s and read 2,071 at 281 tok/s.
The crew scored 8/8 on tests written by gpt-5-mini, matching the OpenAI gpt-5-mini run (8/8) at 2 min 19 s versus 1 min 41 s. Two other runs scored 7/8, both missing fractional refill. Prompt reading was about 300 tok/s on a 1,500-token prompt, 242 at 7.5K, 227 at 9.3K, and 202 at 16.8K.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B at 198.6 tok/s generation and 3657.8 tok/s prompt processing on an M5 Max 128GB.
Setup is oMLX 0.7.1dev2 with oQ5e quant, Lightning MTP speculative decoding, and TurboQuant off, at 1024-token prompt and 128-token generation.
Splash engine reaches 182.0 tok/s generation and 6040.8 tok/s prompt processing on the same test; continuous batching at 4x reaches 275.2 tok/s aggregate generation. A second model, Qwen3.7-27B, is also benchmarked.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at a median 79 tok/s generation on a single RTX 3060 12GB with 32GB DDR5.
Setup is llama.cpp with Q4_K_M and q8_0 KV cache, using --cpu-moe with a 7500MiB MoE expert cache, mlock, and MTP speculative decoding at 32768 context. Prompt processing is around 800 tok/s under 1k tokens, ~1100 tok/s for 1k-4k, and ~1190 tok/s for 4k-16k. MTP draft acceptance is around 85% and VRAM sits at ~11.4GB.
The 79 tok/s median comes from 433 responses longer than 500 tokens, with most between 76 and 83 tok/s. A fixed 3.6k prompt / 700 token output test showed 50-58 tok/s with --n-cpu-moe 25, 71-73 tok/s with --cpu-moe plus cache, 75-81 tok/s with mlock, and 77-84 tok/s after updating llama.cpp. Forcing 4 experts reached 91-95 tok/s but caused looping on long outputs.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
codingtool-uselong-context
User reports Qwen3.6-35B-A3B at 25-35 tok/s on an RTX 4050 Laptop 6GB with 64GB DDR5 RAM and an i5-13420H.
Setup is LM Studio with Q5_K_M and RAM offload, running 34-36k context windows.
User notes a 2-minute time to first token and says the speed is outstanding for the hardware. They ask whether the model and quant are a good fit for a 3D model parameter data structuring workflow, and mention they do not need uncensored or abliterated variants.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
coding
User reports Qwen3.6 35B A3B coder at 24.5 tok/s generation and ~118 tok/s prompt processing on a £299.99 Lenovo IdeaCentre Mini with Snapdragon X1-26-100, 32GB RAM, CPU only.
Setup is llama.cpp with Q4_0 quant, 512 token prompt, 8 threads. The GPU is an Adreno X1-45 and the NPU is unused, so this is all CPU.
With MTP the coder reaches 29-33 tok/s. A custom ARM port of Janas (MoE expert streaming from NVMe) gives 17.2 tok/s plain and 29-31 tok/s with MTP for the coder, and runs an 80B model at 9.8 tok/s plain and 17-18 tok/s with MTP. Gemma 26B runs at 29.5 tok/s in llama.cpp and 23.5 tok/s in Janas. The user also squeezed the coder's expert cache to 12GB and still got 24 tok/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at a little over 43 tokens/s on an RTX 5060 Laptop 8GB in a Lenovo Legion 5 with a Ryzen AI 7 350 and 32GB DDR5-5600.
Setup is llama.cpp with the model offloaded across the 8GB VRAM and system RAM; around 30K context looks promising.
User is also building PolyStrata to try for more performance, with no numbers yet.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 389.0 t/s decode and 5,310 t/s prefill on a single RTX 5090.
Setup is the NInfer engine (ninfer-ext fork) with no quant code stated, one model at a time, no Python in the inference path.
Decode figures are aggregate rates for the stated number of lanes in flight, not per-request latency; the C=1 prefill figure is a true per-request rate while C=2 and above are point aggregates that include each request's decode time. MTP rows draft 3 tokens with the full proposal head and reach 650.3 t/s at C=1, 958.0 at C=2, 1,163.9 at C=4 and 1,368.0 at C=8. The same chart also covers Gemma 4 26B-A4B, Gemma 4 31B IT M1, Qwen3.6-35B-A3B nvfp4, Qwen3.8-27B groupwise-int, Qwen3.8-27B nvfp4 and Qwen3.8-Flash-Next nvfp4.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 27B at 38 t/s generation on an unlocked NVIDIA CMP 170HX 64GB.
Setup is llama.cpp with Q4_K_M and flash attention, full GPU layer offload, zero context depth.
Prompt processing measured 907 t/s at pp512 and 888 t/s at pp8192, five repetitions each; the model file is about 16GB.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 51.7 t/s generation and 1020.8 t/s prompt processing on a Minisforum MS-S1 Max with a Radeon 8060S (gfx1151).
Setup is llama-bench with the ROCm backend, UD-Q4_K_XL quant, ngl 99 and microbatch 2048, on 96 GiB of GPU-accessible unified memory.
The 51.7 t/s is a separate TG128 phase, not generation at the prefilled prompt depths; the prompt figures are a work-size sweep from PP512 to PP32768.
M3 Ultra 512GB · llama.cpp b49650adb31f2e49a0d76113aeb1792134fd8413 / Metal
generation:
81.2 tokens/s
prompt processing (prefill):
2324.6 tokens/s
quant:
Q8_0 (GGUF)
kv:
F16
flash attention:
on
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B Q8_0 at 81.2 t/s generation and 2324.6 t/s prompt processing on a Mac Studio with an Apple M3 Ultra and 512 GiB unified memory.
Setup is llama.cpp with Metal GPU offload, Flash Attention on, F16 KV cache, 8 threads, and batch/ubatch 512/512; the Mac-only control arm uses one GPU.
PP4096 and TG128 are separate synthetic tests, with TG128 starting from an empty context rather than generation after prefill. The article also reports a continuous 4096-input plus 128-output workload at 3.409 seconds, and a separate tuning pass at the same batch/ubatch gives PP4096 2323.37 and TG128 80.68 t/s.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 73.4 t/s generation and 1094.5 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is llama.cpp build 9747 with Vulkan RADV, Q4_K_M quantization and f16/f16 KV cache at depth 1024, one Radeon 8060S GPU.
The headline figures are separate synthetic PP512 and TG128 tests, not one request; generation falls to 58.824494 t/s at depth 32768 with the same KV type.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B Q4_K_M at 60 t/s generation and 960 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is llama.cpp build a66d50588b105ef81fe6c2debd19440f5a8ab97a on ROCm with f16/f16 KV cache at depth 1024, one Radeon 8060S GPU, 8 threads, batch 1024 and microbatch 512.
The headline figures are separate synthetic tests, PP512 and TG128, not one request; the exact values are 959.991546 t/s and 60.021451 t/s, each from a single repetition.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-27B at 12 t/s generation and 322 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory.
Setup is llama-bench with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention; the 120 GB GPU-memory cap is a software limit, not observed usage.
The 12 t/s and 322 t/s are separate 128-token generation and 512-token prompt microbenchmarks, not one end-to-end request; the narrative cites 12.03 t/s while the table rounds to 12.0.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 59.6 t/s generation and 1029 t/s prompt processing on a single Radeon 8060S iGPU in an AMD Strix Halo system with 128 GB unified memory.
Setup is llama-bench with the Unsloth UD-Q4_K_XL quant, full offload, batch 2048 and flash attention; the 20.8 GiB figure is artifact size, not measured resident memory.
The rates are separate 512-token prompt and 128-token generation microbenchmarks, not one end-to-end request, and no TTFT is reported.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 27B at 359.4 t/s prompt processing on an AMD Ryzen AI Max+ 395 with Radeon 8060S graphics.
Setup is llama.cpp b9843 with UD-Q8_K_XL, f16 K/V cache and flash attention, one participating Radeon with ngl=999.
The figure is the stock batch2048/ubatch512 arm at depth 512, a PP-only synthetic run with no generation measured; the same arm falls to 124.740378 t/s at depth 100000.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B-A3B at 1044.1 t/s prompt processing on an AMD Ryzen AI Max+ 395 with Radeon 8060S graphics.
Setup is llama.cpp b9843 with UD-Q8_K_XL, f16 K/V cache, flash attention on, and one participating Radeon.
The figure is the stock batch2048/ubatch512 run at depth 512, measured as 1044.087853 +/- 3.814818 t/s; generation throughput and TTFT were not measured.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6 at 35 t/s generation and 356.8 t/s prompt processing on an AMD Strix Halo 128GB machine.
Setup is the Nathanw1014 Strix Halo llama.cpp v0.6.10 fork with UD-Q4_K_M, f16/f16 KV cache, flash attention on, 16 CPU threads, ngl999 and Vulkan0, at a depth of 131072 tokens.
The figures are native llama-bench pp16384/tg256 results, N=3, with MTP disabled and reasoning off; deeper cells gave 238.8/28.8 at 200000 and 167.4/24.9 at 262144.