Qwen3.8 27B
on M5 Pro 48GB · TensorSharp · 65,536 ctx
Use cases
codingagentic
Summary
User reports running Qwen3.8-27B (UD-IQ3_XXS GGUF) with a DFlash2 Q4_K_M drafter through TensorSharp on an M5 Pro with 48 GB unified memory, decoding at roughly 20-43 tokens/s.
Setup is TensorSharp's OpenAI-compatible server on the ggml_metal backend with a q8_0 KV cache, 65,536-token context and 7 draft tokens per step, driving the Pi Coding Agent.
In the SDK end-to-end run, 41,962 of 48,166 prompt tokens came from the prefix cache and 2,117 tokens were generated; first-request TTFT was about 13-14 s and later rounds 0.45-3.5 s. When draft tokens were rejected more often, decoding fell back to around 21 tokens/s. The user notes this is one coding task on one machine, not a general benchmark.