llamaperf

Qwen3.8 125B (6B active) Flash-Next

on Nvidia AGX 64GB · llama.cpp

Tone: mixed
Oct 17, 2026
Throughput
9.9 t/s gen · 82.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
TQ3_0 (GGUF)
System RAM
64 GB

Summary

User reports Qwen3.8 Flash-Next at 9.86 t/s generation and 82 t/s prompt processing on an Nvidia AGX 64GB embedded computer. Setup is a recompiled llama.cpp with Gyro format support, running the 108GiB TQ3_0 GGUF at a 50W power envelope. Other models on the same machine: Qwen3.6-35B-A3B Uncensored Q6_K_P at 23 t/s for one request and 43 t/s for two, Qwen3.8-27B Q8_K_XL at 8.3 t/s with MTP, Qwen3.8-27B Q4_K_M at 10.5 t/s with MTP, and Qwen3.6-35B-A3B Heretic APEX at 42 t/s with MTP. User is weighing a 128GiB Framework Desktop and doubts it is a step up.