Qwen3.8 27B
on Unknown GPU · llama.cpp
Summary
User reports Qwen3.8-27B at 32.1 tok/s plain decode on an unnamed Apple Silicon Mac with llama.cpp master.
Setup is llama-server on one Mac, -ngl 99, flash attention on, one slot, no speculative decoding; the target model's quant is not stated (Q8_0 is the DFlash2 drafter's quant).
The post benchmarks llama.cpp PR #29869 for Metal speculative decoding. Serial decode is unchanged by the PR at 32.0 tok/s. With DFlash2 speculation the per-prompt figures rise from 30.2 to 110.0 tok/s on code at temperature 0 and from 16.8 to 62.6 on prose. Figures are 64 generated tokens, median of 5 requests, mean of two server runs.