Qwen3.8 Flash-Next Coder
on Unknown GPU · llama.cpp · 262,144 ctx
Use cases
coding
Summary
User reports running a pruned Qwen3.8-Flash-Next Coder GGUF at 70 t/s prompt processing and roughly 8-9 t/s generation on an 8 GB GPU with 24 GB system RAM.
Setup is llama.cpp with a 3.5 BPW GSQ RCO quant, 262K context, most MoE experts kept in RAM and the n-gram table on disk, with selective MoE offloading by regex rather than --n-cpu-moe.
The generation figure is given as a range (8~ T/s) and is limited by PCIe 3.0; the user notes a first run is slower due to MMAP and a second run is needed for accurate numbers. Half the experts are removed, giving an effective 1.89 bits per parameter over the original model.