289 reports. These are individual setups, not a controlled benchmark. Qwen3.6 speeds by hardware →
Unknown GPU
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports a CPU-only test of MiniCPM5 2B. The model is described as much more impressive than MiniCPM5 2B, which was also tested but not extracted as primary.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports prefill around 1000 PP/s and decode 50-70 t/s with Qwen 3.6 35B-A3B on an RTX 5080 eGPU. User asks about CPU instruction set differences for offloading.
M5 Max · MLX
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6 35B-A3B at 130-140 t/s on an M5 Max at 4-bit. Setup uses MLX and llama.cpp engines in a local AI app for macOS.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6 27B at ~24 t/s on dual RTX 3060 GPUs. The user also discusses diffusion-based techniques, MOE, quants, and token authority as a wishlist for future local models.
NVIDIA RTX 3090 · 131,072 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares reported token rates across several local rigs, with Qwen3.6 27B on a single RTX 3090 at about 50 tok/s generation and about 950 tok/s prompt processing. That rig runs a Q4/Q8 MTP configuration at 128k context. The figures are collected second-hand from a discord group rather than measured by the user, who also lists about 140 tok/s on a 5090 and 80-100 tok/s for DeepSeek V4 Flash on dual RTX 6000 cards.
M5 Max 128GB · MLX · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks a 128 GB M5 Max MacBook Pro across multiple models, selecting Qwen 3.6 27B MLX Q8 as the primary one. Other models tested include Gemma 4, MiniMax M2.7 and DeepSeek V4 Flash. The user is enthusiastic about local AI.
M5 Max 128GB · llama.cpp · 128,000 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks several models on an M5 Max 128GB and selects Qwen3.6 35B A3B as the primary model for its MoE detail. Other models tested include Gemma 4, Qwen 3.6 27B, MiniMax M2.7, Mistral Medium 3.5, Step 3.7 Flash and DeepSeek V4 Flash. Results show strong performance for MoE models.
NVIDIA RTX 5060 Ti 16GB · llama.cpp · 51,200 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B MoE at about 40 t/s on an RTX 5060 Ti 16GB with mmproj enabled, and just over 50 t/s without mmproj and with --n-cpu-moe 24. Setup is llama.cpp server in Docker with --cache-type-k/v q8_0, --cache-ram 0, --no-mmap, and --n-cpu-moe 19, or 24 without vision. The user experienced an OOM shutdown before adding --cache-ram 0 and is seeking advice on the setup.
User reports Qwen 3.6 27B abliterated BF16 running locally via vLLM with llama-swap and MTP speculative decoding, scoring 8/10 on a Terminal-Bench 2.0 pilot. The same setup beats DeepSeek-V4 IQ2 at 7/10 and comes close to an FP8 API at 9/10. It is the only configuration to pass the cancel-async-tasks hard task, and it missed build-cython-ext and sqlite-db-truncate on timeout.
NVIDIA DGX Spark · vllm.cpp
User reports a C++20 port of the vLLM serving stack on a DGX Spark with Qwen3.6-27B NVFP4, reaching 86.05 to 1095.01 output tokens/sec across concurrency 1 to 32. The port runs slightly ahead of vLLM in the same tests. The user also reports DeepSeek-V4-Flash in 2-bit GGUF at 18.69 tok/s on the Spark, and compares against llama.cpp and MLX-LM.
AMD Radeon AI PRO R9700 32GB · vLLM · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6 35B-A3B MoE on a single R9700 with vLLM, using INT4 weights from Avesed. The user also tests a 27B dense model with MTP spec=4. Prefill and decode speeds are measured at various context depths, and the user is happy with the results.
AMD Radeon 780M · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6 35B-A3B Q8_0 on a Ryzen 7 260 with a 780M iGPU and 64 GB DDR5, reaching pp8192 287.33 t/s and tg128 21.06 t/s. Setup is the Vulkan backend. Gemma 4 31B Q8_0 on the same hardware gives pp8192 51.59 t/s and tg128 2.46 t/s. With MTP, Gemma 4 31B reaches tg ~5.76 t/s, and Qwen3.6 35B-A3B with MTP and partial offloading reaches tg ~34.85 t/s. The user mentions an RTX 5060 8GB as a bonus for MoE partial offloading.
NVIDIA CMP 170HX 64GB (unlocked) · llama.cpp · 262,144 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User reports Qwen3.6-35B-A3B at 63 t/s on 4x CMP 170HX 8GB cards flashed to 64GB each, 256GB total. Setup uses MTP with little-MoE default. Generation reaches 110 t/s with MTP optimistic.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen 3.6 35B-A3B at about 70 t/s on an RTX 3060. The figure is given as past experience rather than a benchmark run.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks Qwen3.6-35B-A3B on a Mac with OMLX, comparing MTP enabled against disabled. MTP shows minimal speedup for the 35B MoE model but roughly 2x for the 27B dense model. Results include pp and tg t/s at various context lengths and batch sizes.
NVIDIA RTX 5070 Ti Laptop 12GB · llama.cpp · 4,096 ctx
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User compares Qwen3.8 27B dense at Q2 and Q3 against Qwen3.6 35B-A3B MoE on 12 GB of VRAM. The MoE was fastest and passed a sanity test, while dense Q3 ran at 7.5-9.1 t/s. The user prefers the MoE for local use.
AMD Radeon 780M · llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing.
User benchmarks ROCm against Vulkan on a Radeon 780m iGPU. ROCm gives a 50% pp speedup for dense models. The user also reports Qwen3.8 27B results.
NVIDIA RTX Pro 6000 Blackwell · Paddock
User benchmarks Qwen3.6-27B at 32 concurrent clients on the Paddock engine, with TTFT of 697 ms against 2.5 s for vLLM and 6.9 s for llama.cpp. User also reports Qwen3.8-27B at Q8 on an RTX PRO 6000 with speculation on and off: 46.8 to 202 tok/s single stream, 320 to 822 at eight concurrent chats, and 1005 to 1285 at 32. The engine is free but not open source.
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B at 39 t/s on an 8 GB RTX 4060 laptop using the FreeToken engine. User also reports DeepSeek-V4-Flash 284B at 22-25 t/s on an RTX 5090 and GLM-5.2 753B at 15 t/s on an RTX PRO 6000.
RTX 4000 · ik_llama.cpp
Reported by the source; GPU count, offloading and concurrent requests can change these figures. Prompt-processing speed is shown only when the source reports it; generation speed cannot tell us time to first token. Check the full setup before comparing.
User reports Qwen3.6 35B Q6 at 35-40 t/s decode on an RTX 4000 in a VM. User is considering swapping to a Strix Halo (Bosgame M5) for more concurrent models or a larger MoE. User also mentions Qwen3.8 27B on an RTX 3090 desktop and plans for larger models such as 3.5 122B or 3.8 Next Flash.