llamaperf

Qwen3.8 125B (6B active) Flash-Next

on AMD Strix Halo 128GB · llama.cpp · 262,144 ctx

AMD hardware
Oct 12, 2026
Throughput
15.6 t/s gen · 109.6 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
UD-Q4_K_XL (GGUF)
KV cache
F16
Flash Attention
on
MTP (Multi-Token Prediction)
off
System RAM
128 GB

Summary

User reports Qwen3.8 Flash-Next at 15.6 t/s generation and 109.6 t/s prefill on an AMD Strix Halo 128GB workstation. Setup is llama.cpp with UD-Q4_K_XL, F16 KV cache and flash attention, 40 GPU layers offloaded in a single inference slot with 2048 MiB prompt-cache capacity, configured to 262144 context. CPU-backed per-layer token embeddings make this a hybrid offload rather than a full-GPU test, and actual input/output lengths and cache hits are unreported.