Qwen3.8
on Unknown GPU · llama.cpp
Summary
User reports Qwen3.8 in llama.cpp with MTP combined with ngram-mod speculative decoding on a coding task that repeats a code block 10 times.
Output came in bursts, starting at up to 169 t/s and degrading over the repetitions to about 80 t/s average. Prompt eval was 982.53 t/s and generation 80.30 t/s, with draft acceptance 0.85192.
User asks whether parameter tuning would help and whether the ngram combination is worth it.