Qwen3.8 27B
on NVIDIA RTX 3090 · llama.cpp · 150,000 ctx
Use cases
codinglong-context
Summary
User compares llama.cpp with a Q4_K_XL quant against vLLM with W4A16-AutoRound on an RTX 3090.
llama.cpp reaches about 45-50 t/s at 150K context, while vLLM reaches about 55-65 t/s at 175K context.
The user prefers vLLM for speed but notes llama.cpp is needed for vision, and asks about the quality difference between the quants for coding.