llamaperf

Qwen3.8 Flash-Next Coder

on Unknown GPU · llama.cpp · 262,144 ctx

Tone: positive
Oct 24, 2026
Throughput
70.0 tokens/s prompt processing (prefill)Prompt processing measures input prefill speed; it does not tell you the time to first token.
Quant
3.5 BPW GSQ RCO (GGUF)
System RAM
24 GB
VRAM reported
8 GB

Use cases

coding

Summary

User reports running a pruned Qwen3.8-Flash-Next Coder GGUF at 70 t/s prompt processing and roughly 8-9 t/s generation on an 8 GB GPU with 24 GB system RAM. Setup is llama.cpp with a 3.5 BPW GSQ RCO quant, 262K context, most MoE experts kept in RAM and the n-gram table on disk, with selective MoE offloading by regex rather than --n-cpu-moe. The generation figure is given as a range (8~ T/s) and is limited by PCIe 3.0; the user notes a first run is slower due to MMAP and a second run is needed for accurate numbers. Half the experts are removed, giving an effective 1.89 bits per parameter over the original model.