Qwen3.8 125B (6B active) Flash-Next
on AMD Strix Halo 128GB · llama.cpp · 64,000 ctx
Summary
User reports Qwen3.8-Flash-Next-UD-Q5_K_XL on Strix Halo 128GB with mmproj, MTP, a 64000 Q8 KV cache and 2GB cache-ram, leaving only 1-2GB free memory.
Q4_K_XL (77GB) and Q5_K_XL (108GB) were also tested. MTP adds 5.5GB even when the file is 2.8GB. F16 KV cache uses 1GB per 10000 tokens; Q8 uses 1GB per 20000 tokens.
Perplexity results: Q5_K_XL Top-1% 94.795, Mean KLD 0.018175, 99.9% KLD 0.591240; Q4_K_XL+50GB-NGRAM Top-1% 93.786, Mean KLD 0.027853, 99.9% KLD 0.948222; Q4_K_XL+25GB-NGRAM Top-1% 93.285, Mean KLD 0.033378, 99.9% KLD 1.090875. The F16 base-kld could not be run due to insufficient memory.