llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 5× M4 Pro 24GB · 60,000 ctx

Tone: positiveApple hardware
Oct 18, 2026
Throughput
37-39 t/s gen · 856-895 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
3-bit

Use cases

codingagenticlong-context

Summary

User reports Qwen3.8 Flash-Next at 37-39 tok/s writing and 856-895 tok/s reading the prompt at 60k context, split across five devices: an RTX 2060 Super PC, an RX 9060 XT PC, an M4 Pro MacBook with 24 GB, and two iPhone 17 Pro Maxes (A19 Pro). Setup chains the devices over Ethernet and 10 Gb/s USB-C, with every weight held in VRAM, pinned RAM or unified memory and no disk streaming. Pipelined speculation from the model's own MTP draft head lands about 2.2 tokens per step. At 16k context the user measured 42 tok/s writing and 728-744 tok/s prefill; at 32k, 39-41 tok/s and 844-852 tok/s. The agent session averaged 35 tok/s with bursts past 40. A second layout with 8-bit KV on the Mac and phones writes 31-33 tok/s at 60k and read one prompt past 164k tokens. PC 2's 16 GB caps prefill at about 980 tok/s.