llamaperf

Qwen3.8 125B (6B active) Flash-Next

on 2× M5 Ultra 256GB · Tensorfold

Tone: mixedApple hardware
Oct 26, 2026
Throughput
200.0 t/s gen · 8000.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
Q6
System RAM
256 GB

Use cases

agenticcoding

Summary

User reports Qwen3.8 Flash-Next at 200 tok/s decode and 8k tok/s prefill on a two-machine M5 Ultra 256GB (80-core) cluster. Setup is Tensorfold with Q6 weights, connected over two Thunderbolt 5 cables; the user copied Tensorfold's guide and had DeepSeek set it up. GLM-5.3 Flash on the same cluster gave 50 tok/s decode and 4.9k tok/s prefill, which the user found too slow. A previous Qwen3.8 27B Q4 run under NInfer gave around 1.4k prefill and 100-120 tok/s decode. The user runs Hermes as the harness and asks whether a lighter harness would speed things up.