llamaperf

Qwen3.8 27B

on M5 Pro 48GB · TensorSharp · 65,536 ctx

Apple hardware
Nov 11, 2026
Throughput
20-43 t/s gen
Quant
UD-IQ3_XXS (GGUF)
KV cache
q8_0
System RAM
48 GB

Use cases

codingagentic

Summary

User reports running Qwen3.8-27B (UD-IQ3_XXS GGUF) with a DFlash2 Q4_K_M drafter through TensorSharp on an M5 Pro with 48 GB unified memory, decoding at roughly 20-43 tokens/s. Setup is TensorSharp's OpenAI-compatible server on the ggml_metal backend with a q8_0 KV cache, 65,536-token context and 7 draft tokens per step, driving the Pi Coding Agent. In the SDK end-to-end run, 41,962 of 48,166 prompt tokens came from the prefix cache and 2,117 tokens were generated; first-request TTFT was about 13-14 s and later rounds 0.45-3.5 s. When draft tokens were rejected more often, decoding fell back to around 21 tokens/s. The user notes this is one coding task on one machine, not a general benchmark.