Qwen3.8 27B
on 2× NVIDIA RTX 3090 · Geezuz
Use cases
coding
Summary
User reports Qwen3.8-27B at 190 tok/s on two RTX 3090s power-limited to 280 W over plain PCIe, at 0 context.
Setup is the Geezuz engine, a dual-GPU fork of L-Forster's open-jet megakernel, with Q4_K_M GGUF weights and 8-bit KV cache; the model drafts with its own MTP head, verifying 5 drafted tokens per pass.
At 8 concurrent requests the total is 622 tok/s; Q8_0 gives 143 tok/s single and 610 tok/s at 8. The user says llama.cpp's fastest two-GPU setup did about 100 tok/s on the same prompts, and that Geezuz averaged 164 tok/s and 214 on code at the same KL divergence. At 10K context it reports 159 tok/s single-request and 550 at 8 requests; vLLM still leads at 8 parallel requests by about 15%.