Qwen3.8 27B
on 4× NVIDIA RTX 3090 · llama.cpp · 400,000 ctx
Sep 17, 2026
Use cases
agenticlong-context
Summary
User reports Qwen3.8 27B at ~70 t/s decode on an RTX 3090 plus RTX 5070 Ti, with a ~400K shared context pool.
Setup is llama.cpp with the IQ4_XS GGUF, MTP and tensor parallelism across 3 cards at np 4, allowing up to 4 concurrent agents. A Q6_K_M quant of the same model also runs on the 3090 and 5070.
The user asks whether other engines handle mixed Nvidia GPUs better, noting vLLM, SGLang, and Freetoken did not work well for this rig.