llamaperf

Qwen3-Next

Alibaba · 9 reports

Qwen3-Next 80B · 3B active at 4-bit generates a median of 2.9 tokens per second on the M4 Pro 24GB, the hardware it is most run on, from one run that kept part of the model in system RAM (community reports, as of 22 Nov 2026).

As of 22 Nov 2026, Qwen3-Next 80B · 3B active at 4-bit on each device, fastest first, with the median generation speed of runs that kept part of the model in system RAM (one device, one request):

Bars are medians on one scale, 0 to 94.5 tokens/s. Press ▶ to watch text arrive at a row's speed.

The same figures as a table, with prompt processing →
Qwen3-Next VRAM requirements by size and quant →

How does Qwen3-Next run on your hardware?

Pick your GPU or Mac, then read the speeds people reported for Qwen3-Next on it, or estimate memory fit and speed.

Free to use. No account needed. Memory estimates and community measurements are labelled separately.

Run Qwen3-Next yourself? Add your numbers.

Your coding agent can run a speed test on your machine and send it here with a free account. Or paste a result you've already posted.

Reported performance for Qwen3-Next

Filter this model’s reports by setup →

Speeds are as each source reported them. GPU count, offloading and concurrent requests change them, and prompt-processing speed is input throughput, not time to first token. Check the full setup before comparing two reports.

Qwen3-Next 80B (3B active)

Arc iGPU · Janas-LLM

generation:
26.4 tokens/s
prompt processing (prefill):
68.4 tokens/s

User reports Qwen3-Next-80B-A3B at 26.4 tok/s generation and 68.4 tok/s prompt processing on Janas-LLM, against 3.0 and 17.7 tok/s on llama.cpp. Setup is Janas-LLM streaming MoE experts from NVMe on a Core Ultra 9 185H laptop with 32 GB DDR5-5600 and an Arc iGPU; the 48.4 GB model is larger than the RAM. The same table gives Qwen3.6-35B-A3B at 28.0 tok/s reply and 136 tok/s prompt on Janas-LLM versus 12.1 and 81.8 on llama.cpp, and Qwen3-30B-A3B at 33.3 and 133 versus 15.0 and 122. The post announces Brebus, a Linux desktop app built on the engine.

Nov 16, 2026

Qwen3-Next 80B (3B active)

Intel Arc iGPU · Janas-LLM · 1,024 ctx

Tone: positive
generation:
26.4 tokens/s
prompt processing (prefill):
68.4 tokens/s

User reports Qwen3-Next-80B-A3B at 26.4 tok/s generation and 68.4 tok/s prompt processing on a laptop with an Intel Core Ultra 9 185H, 32 GB DDR5, integrated Arc GPU and NPU, and NVMe Gen4. Setup is the author's Janas-LLM engine in plain C, streaming MoE experts from NVMe with an expert cache in RAM; replies run on the CPU, prompts on the iGPU or NPU. The 48.4 GB model is larger than system RAM. llama.cpp on the same files reached 3.0 tok/s generation and 17.7 tok/s prompt on this model. The post also tables nine other models, MTP drafts giving up to 63% faster chat replies on Qwen3.8-Flash-Next, and 18.2 tok/s generation at 8,192 tokens on Qwen3-4B with an 8-bit KV cache.

Oct 29, 2026
NVIDIA hardware
generation:
67.5 tokens/s
quant:
Q4_K_M (GGUF)

User benchmarks Qwen3-Next-80B-A3B-Instruct Q4_K_M on an RTX 4070 Ti Super 16GB, reporting 67.54 t/s median decode on real text and 80.673 t/s on a repeated prompt. Setup is llama.cpp b11617 with -ncmoe 48, --moe-cache-mib 10752, 4096 context and a single-CCD CPU affinity mask; the 46.64GB model streams experts from system memory over PCIe. All figures are warm-cache, 435 prompt tokens and 256 generated tokens. A 512 MiB expert cache was 23.8% slower than no cache, and decode fell 70.5% at -ncmoe 44. Random token IDs decoded at 93.61 t/s, about 38% faster than real text.

Oct 19, 2026
Tone: mixedApple hardware
generation:
2.9 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3-Next 80B-A3B at 2.9 tok/s on an M4 Pro 24GB MacBook Pro, running llama.cpp's WebGPU backend in a Chrome tab with MoE experts paged in from disk. Setup is llama.cpp (ggml-webgpu) with a Q4_K_M GGUF, 48.4 GB file, 4.89 GB resident on the GPU, 48 s initial load from disk. The 80B reads about 359 MB from disk per token; the author calls it a proof of concept. OLMoE 1B-7B Q4_K_M reached 22.4 tok/s and LFM2 8B-A1B Q4_K_M 17.1 tok/s in the same recording. Paged and unpaged replies were identical; against llama.cpp on Metal 4 to 8 of 9 replies matched. Chrome or Edge 137+ only, tested on Apple silicon only.

Oct 14, 2026
Apple hardware
generation:
49.1 tokens/s
quant:
4-bit (MLX)

User reports 49.1 t/s generation. Source evidence (verbatim): Source: https://arxiv.org/html/2605.00519v1 # First-hand M3 Ultra 96GB Qwen3-Next 80B native MLX 4-bit generation measurement Original experimental paper Silicon Showdown, arXiv 2605.00519v1, May 2026. Selected Table 5 row: Apple M3 Ultra 96GB, 28 CPU cores, 60 GPU cores; installed unified memory 96GB. macOS 26 Tahoe, MLX v0.30.6. Qwen3-Next-80B-A3B native MLX 4-bit: generation 49.1 tokens/s, 0% CPU offload. Long prompt approximately 496 tokens; output 256 tokens, greedy decoding. Authors report arithmetic mean of three consecutive timed runs after two untimed warmups and fixed random seed. This is their published generation-phase measurement, not a speed derived from TTFT. Prompt-processing speed unreported; leave promptTps null. Context capacity, KV quantization, speculation, concurrency and resident allocation unreported. Verified base-model reference explicitly cited by the paper: https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct . This is not the tested MLX quantization repository; exact quantized artifact and revision unknown. Do not infer disk size or resident memory. Table 2 names the 96GB machine, not the followed 256GB configuration. Other models, machines and Nvidia offload comparisons are separate runs. Source: https://arxiv.org/html/2605.00519v1 .

Oct 14, 2026
Tone: positiveNVIDIA hardware
generation:
94.5 tokens/s
quant:
Q4_K_M (GGUF)

User reports Qwen3-Next-80B-A3B at 94.5 tok/s decode on an RTX 3090 with 16 GB RAM, using a patched llama.cpp that substitutes missing experts instead of waiting for SSD reads. Setup is llama.cpp with Q4_K_M, 1/4 of experts in VRAM, rest read from NVMe at ~5.7 GB/s, 16 threads, about 15.7 GB VRAM used. Stock llama.cpp gave 31.8 tok/s in the same 16 GB case; the patch also reached 108.4 tok/s with plenty of RAM and 89 tok/s reading every miss from SSD. Perplexity was 1.6% higher than stock, GSM8K lost 1.8 points, and greedy generation ran 64-74 tok/s. The 16 GB case was simulated by locking RAM on a bigger machine.

Oct 6, 2026

Qwen3-Next 80B (3B active)

Unknown GPU · llama.cpp · 32,768 ctx

Tone: positive
generation:
19.1 tokens/s
prompt processing (prefill):
130.0 tokens/s
quant:
Q4_K_M (GGUF)
coding

User reports Qwen3-Next-80B-A3B at 130 t/s prefill and 19.1 t/s generation on a CPU-only Lenovo ThinkStation P5 with Intel Xeon w5-2555X and 256 GB DDR5 ECC. Setup is llama.cpp built with GGML_NATIVE=ON, Q4_K_M quant, 32k context, mlocked into RAM, no GPU offload. The user also benchmarks Qwen3-Coder-30B-A3B at 166 t/s prefill and 33.7 t/s generation, gpt-oss-120b at 98 t/s prefill and 19.5 t/s generation, and Qwen3-235B-A22B at 24.8 t/s prefill and 5.6 t/s generation. An NVIDIA T1000 8GB was tested and found to slow prefill versus CPU-only, so it was removed from inference. The user notes that -ub tuning varies per model and that Q8 quant of Qwen3-Next-80B scored 59/61 on a coding suite versus 55-57 for Q4_K_M.

Oct 3, 2026

Qwen3-Next Flash

6× BC-250 · llama.cpp · 100,000 ctx

Tone: positive
generation:
28.0 tokens/s
quant:
IQ2_XS (GGUF)

User reports Qwen3-Next Flash IQ2_XS at around 28 tok/s for short generation on a cluster of 4 BC-250 boards, dropping to 24 tok/s at 50k context with around 115 t/s prefill. Setup is llama.cpp with Vulkan and RPC over 1Gb Ethernet, 100k context, across 6 BC-250 ex-mining boards in an ASRock 4U12G case. The other two boards run Qwen3.6 35B Q4 at 60 tok/s with 100k context and 450 t/s prefill.

Oct 2, 2026
NVIDIA hardware
generation:
116.0 tokens/s
prompt processing (prefill):
6800-7950 tokens/s
quant:
W4A16

User reports Qwen3-Next-80B at 116 t/s single-stream decode on a single NVIDIA CMP 170HX with 64 GB HBM2e, at a 150 W cap. Setup is vLLM 0.27.1 with W4A16 weights (40.9 GB), torch 2.13.0+cu130, CUDA 13.0, Ubuntu 26.04 LTS, on an AMD Ryzen Threadripper PRO 3945WX with 128 GB DDR4 ECC. Prefill over about 8.9k tokens measured 6800 to 7950 t/s; power draw 137 to 145 W. Aggregate throughput at 8 concurrent requests was 352 t/s, saturating at 4 slots. Also measured on the same card for context: Ornith-1.5-35B FP8 at 122.5 t/s and Qwen3.8-27B W4A16 with DFlash2 at 127 t/s single-stream.

Sep 23, 2026
Engines people run Qwen3-Next with
EngineReports
llama.cpp5
MLX1
vLLM1

Get a weekly email of new Qwen3-Next reports on any GPU.

Email me new reports

Reported configurations

Individual observations from this page, not expected speeds or a ranking. Select a model to open its report, with the source and offloading conditions. Reported context may be a limit rather than actual input length.

Model / hardwareQuant / engineReported contextGeneration
Qwen3-Next 80B (3B active)
NVIDIA RTX 4070 Ti Super
Q4_K_M
llama.cpp
4,09667.5 tokens/s
Qwen3-Next 80B (3B active)
M4 Pro 24GB
Q4_K_M
llama.cpp
Not reported2.9 tokens/s
Qwen3-Next 80B (3B active)
M3 Ultra 96GB
4-bit
MLX
Not reported49.1 tokens/s
Qwen3-Next 80B (3B active)
NVIDIA RTX 3090
Q4_K_M
llama.cpp
Not reported94.5 tokens/s
Qwen3-Next 80B (3B active)
NVIDIA CMP 170HX 64GB (unlocked)
W4A16
vLLM
Not reported116.0 tokens/s

With part of the model in system RAM: Qwen3-Next 80B · 3B active at 4-bit

Runs on one device that kept some of the model in system RAM or streamed it from an SSD. The CPU and its memory set the pace as much as the card, so these figures are kept apart from runs with the whole model on the device and vary more from machine to machine.

HardwareGen t/sPP t/sRuns
NVIDIA RTX 309094.5-1
NVIDIA RTX 4070 Ti Super67.5-1
M4 Pro 24GB2.9-1

Frequently asked

What hardware do people run Qwen3-Next on?

The hardware Qwen3-Next 80B · 3B active at 4-bit is most run on: M4 Pro 24GB (2.9 t/s median, one run); NVIDIA RTX 3090 (94.5 t/s median, one run); NVIDIA RTX 4070 Ti Super (67.5 t/s median, one run). Medians of runs that kept part of the model in system RAM, as of 22 Nov 2026.

Which inference engines do people use for Qwen3-Next?

Reports of Qwen3-Next name llama.cpp (5 reports), MLX (1 report), vLLM (1 report), as of 22 Nov 2026.