Qwen3.8 27B
on 48 GB VRAM GPU tier · llama.cpp · 262,144 ctx
Use cases
summarizationlong-context
Summary
User compares self-hosted Qwen3.8-27B Q8_0 on a RunPod 48GB GPU tier at about $1.22/hr against OpenAI GPT-5.6 Sol for a 47k-token transcript analysis task.
Prompt processing runs at about 1073 t/s and generation at about 17.2 t/s in llama.cpp, which warns about unused NextN/MTP tensors (blk.64.nextn.*), suggesting speculative decoding is not active.
User asks whether switching to Q4_K_M or Q5, reducing context to 64k, enabling MTP, using vLLM, or using a 5090, L40S, A100 or H100 would help.