llamaperf

Qwen3.8 27B

on 4× NVIDIA RTX 3090 · llama.cpp · 400,000 ctx

Tone: positiveNVIDIA hardware
Sep 17, 2026
Throughput
70.0 t/s gen
Quant
IQ4_XS (GGUF)
MTP (Multi-Token Prediction)
on
System RAM
64 GB
VRAM reported
72 GB

Use cases

agenticlong-context

Summary

User reports Qwen3.8 27B at ~70 t/s decode on an RTX 3090 plus RTX 5070 Ti, with a ~400K shared context pool. Setup is llama.cpp with the IQ4_XS GGUF, MTP and tensor parallelism across 3 cards at np 4, allowing up to 4 concurrent agents. A Q6_K_M quant of the same model also runs on the 3090 and 5070. The user asks whether other engines handle mixed Nvidia GPUs better, noting vLLM, SGLang, and Freetoken did not work well for this rig.