Local model performance on your hardware

generation:

32.6 tokens/s

prompt processing (prefill):

622.0 tokens/s

quant:

UD-Q6_K_XL (GGUF)

flash attention:

on

User reports Qwen3-Coder-Next at 32.6 t/s generation and 622 t/s prompt processing on a Radeon 8060S Graphics (gfx1151) under ROCm 7.1.1 on Ubuntu 24.04. Setup is llama-bench with UD-Q6_K_XL, ngl 99, FA enabled, 4096 context depth, 2048 prompt and 32 generation tokens, plus a 512 MB BIOS VRAM carve-out and GTT configuration. The figures are separate synthetic prompt-processing and generation phases, not a single request; at 4096 depth the same run measured 590.84 t/s prompt and 32.08 t/s generation.

prompt processing (prefill):

698.2 tokens/s

quant:

UD-Q8_K_XL (GGUF)

kv:

F16

flash attention:

on

User reports Qwen3-Coder-Next at 698.2 t/s prompt processing on an AMD Ryzen AI Max+ 395 with Radeon 8060S graphics. Setup is llama.cpp b9843 with UD-Q8_K_XL weights, f16 K/V cache and flash attention, one participating Radeon. The figure is the stock batch2048/ubatch512 arm at depth 512, measured as 698.233773 +/- 16.683540 t/s; generation throughput was not measured.

generation:

34.5 tokens/s

quant:

6-bit (MLX)

User reports Qwen3-Coder-Next 80B-A3B at 34.5 tok/s decode on an M5 Pro 48GB Mac, running a full agentic coding task in 44s. Setup is the Global Agent engine (MLX-based) with 6-bit weights and expert streaming from SSD, speed mode, peak memory 31.5GB. The post also benchmarks Qwen3.8-Flash-Next 229B-A7B (19.2 tok/s), Qwen3-235B-A22B (9.19 tok/s) and MiniMax-M3 426B-A24B (4.62 tok/s) on the same machine, plus 16GB and 32GB memory-capped runs; the 16GB Qwen3-Coder-Next run reached 41.2 tok/s. Exact mode figures are lower (17.7, 7.88, 2.62, 2.20 tok/s).

generation:

7.7 tokens/s

quant:

Q4_K_M (GGUF)

User reports Qwen3-Coder-Next (80B MoE, 3B active) at 7.74 tok/s generation on an AMD Ryzen AI 9 HX PRO 370 with 96GB DDR5-5600, CPU-only. Setup is Ollama 0.15.6 (llama.cpp build 7973) with Q4_K_M GGUF weights; 12 physical threads was optimal, SMT hurt, and CPU governor made no difference. User compares against qwen2.5-coder:32b Q4_K_M at 3.54 tok/s on the same hardware and argues the MoE should reach 20-30+ tok/s by bandwidth math; cross-references an EPYC 9454P report where Qwen3-Next-80B hit 11.76 tok/s versus 63.10 for Qwen3-MoE-30B, and notes ik_llama.cpp reports ~1.9x faster MoE inference.

generation:

40.0 tokens/s

quant:

Q4 (GGUF)

User reports Qwen3-Coder-Next Q4 at 40 t/s on 5 ex-mining BC-250 boards with about 71GB of VRAM exposed. Setup uses an ASRock 12-unit case with one board as the main and the rest headless, all over the boards' onboard 1Gb Ethernet. The 40 t/s figure is at 30k context and dips to around 30 t/s at 100k context. User notes the setup is wildly inefficient with power but cost less than $800, and plans to add 2 more boards to test Qwen3.8 Flash-Next.

generation:

24.0 tokens/s

quant:

Q4_K_M (GGUF)

User reports Qwen3-Coder-Next Q4_K_M generating roughly 24 t/s on an M4 Pro with llama.cpp build b7935, compared to roughly 60 t/s for the MLX 4bit version on the same machine. Setup is llama.cpp (llama-server) with Q4_K_M GGUF, 40000 token context, all layers on GPU, parallel 1, continuous batching. User notes the filled context was negligible as the models were freshly loaded, and that the same GGUF in LM Studio was also slow.

generation:

36.8 tokens/s

prompt processing (prefill):

545.8 tokens/s

quant:

UD-Q6_K_XL (GGUF)

kv:

f16

User reports Qwen3-Coder-Next UD-Q6_K_XL at 36.75 t/s on tg32 on an AMD Strix Halo box, with prefill at 545.77 t/s on pp2048. Setup is llama-server with -ngl 999, flash attention on, -c 262144, f16 KV cache and one slot, measured with llama-benchy 0.4.1 in API latency mode. The tg32 rate is 36.75 ± 0.05 t/s with a peak of 37.94 ± 0.06 t/s. A 27B 8-bit XL model was too slow to be workable.