Qwen3.8 27B
on NVIDIA RTX 5090 · llama.cpp · 40,960 ctx
Use cases
codingtool-use
Summary
User benchmarks Qwen3.8 27B across six engines on one RTX 5090 32 GB, with llama.cpp reaching the highest plain single-request decode at 84.2 t/s.
Setup is llama.cpp with an NVFP4 GGUF target and Q4_K_M DFlash2 drafter, 40,960-token context, greedy decoding with thinking off; the plain figure has no speculation. Other engines used different checkpoints and drafters, so results describe whole configurations.
With DFlash2, NInfer led at 280.5 t/s single-request and 488 t/s aggregate across four requests; llama.cpp measured 212.9 t/s and 259 t/s aggregate. The author treats differences below roughly 5% as inconclusive.